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[IMPROVED) -PROGRAM VIEWING APPARATUS AMD METHOD 



BACKGROUND OF THE INVENTION 



Field of the Invention 



This invention relates to the field of delivery of 



programming, for example entertainment and educational 
programming such as that currently delivered by television and 
film technology. The application is a continuation-in-part of 
our prior application U.S. 08/322,069 filed October 12, 1994 for 
a Frequency Convertor System, the contents of which are 
incorporated by reference. ' 

Description of the Prior Art 
In the Prior art it is known to deliver programming 
such as entertainment and educational programming by physical 
shipment of films and magnetic recording, terrestrial 
transmission of television signals via wires or fibers, 
transmission of radio frequency energy via satellite or 
otherwise. In particular, it is known to transmit multiple 
channels of television programming utilizing compressed digital 
or analog data over satellite and cable television systems. For 
all of these modes of transmission a user may typically either 
view the program as it is transmitted or may record a single 
transmitted program for later viewing. With present technology, 
however, it is normally required to have available one recorder 
for each transmitted program. Some in the "prior art have 
attempted to develop a frequency converter system that would 
allow this to change with varying degrees of success. 



Although some of these frequency converter systems 



functioned properly, many of these frequency converter systems 
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were excessively complex and costly to manufacture. Accordingly, 
the frequency converter systems of the past have not found wide 
use in the media art. 

Frequency converter systems of the prior art include 
U.S. Patent 4,829,257 to J. Carl Cooper for an Improved Device 
for Accurately Phase or Frequency Shifting an Input Signal. This 
invention essentially incorporated a variable resistor extending 
between at least two known phase shifted values of the input 
signal. U.S. Patent 4,868,428 to J.Carl Cooper discloses an 
Apparatus and Method for Accurately Shifting the Phase or 
Frequency of a Complex Signal. U.S. Patent 5,097,218 to J. Carl 
Cooper discloses an Apparatus and Method for Accurately 
Multiplying the Phase or Frequency of Complex Time Varying 
Signals by a Given Factor Which May be Non- integer. The contents 
of these patents are incorporated by reference. 

OBJECTS OF THE INVENTION 
It is an object of the invention to provide an 
improved program delivery apparatus and method which is - 
compatible with existing program delivery systems and provides 
the user with improved access to such programming. In 
particular, as described below, it will be possible for each user 
to select and view one and/or multiple programs virtually at 
random (within the limits of storage availability) whenever he 
desires. In addition, it will be possible for a user to stop or 
suspend viewing in the middle of a program (such as to 
accommodate interruption by a phone call which may also be 
provided by the present invention) and to continue viewing after 
the interruption without missing any of the program. During the 
time the viewer is occupied with the phone call the incoming 
program would be recorded. Further, the user can manually and/or 
automatically accelerate programs to accommodate for the time 
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lost and/or otherwise compensate for time without objectionable 
artifacts. In addition, previously stored programs may be viewed 
at accelerated (or decelerated) speed, or repeated or portions 
reviewed, all upon the viewers wish, via the frequency convertor 
for example or otherwise. 

This system provides a user with the ability to change 
the frequency of the produced signal by a percentage function 
based on the ratio between the lengths of time of production. 
Alternately the user can change the frequency of a real time 
signal (i.e., without altering the time period of production). 
For example, it might be desirable to replay a prerecorded thirty 
(30) minute television program in a time duration of twentyeight 
(28) minutes in order to fit a time slot without the associated 
seven percent (7%) increase in frequency. The replay of a 
prerecorded thirty (30) minute television program in twenty seven 
(27) minutes would allow alternately for the three (3) minutes of 
phone calls or other activities during the program. The present 
invention also allows shows to be expanded to fit time slots. 

In another example, entertainment or educational pro- 
grams or movies could be presented in shorter time to reduce the 
operating costs or to allow more movies to be shown in X evening. 
A similar advantage could be realized in the replay of 
prerecorded music or voice on a radio station. Messages from an 
answering machine could also be accelerated, perhaps greatly, for 
rapid playback while retaining normal voice frequencies. Again, 
the present invention allows this without the pitch shift 
artifacts that would otherwise be ascertainable to the consumer. 
Another example would be to lower the occupied bandwidth of a 
signal to be transmitted over a radio propagation or other 
transmission medium. This is a major thrust of this present 
invention. In this specification, the data may be fed into a 
large memory (for example 4-5 gigabytes) , perhaps intermittently, 

G:\7434CIP -3- 06/08/95 



at one speed and fed out at a second (normally .slower and perhaps 
constant) speed, thus facilitating signal presentation and/or 
data operations. Another object of the present invention is to 
provide an improved apparatus and method for frequency conversion 
capable of decreasing or increasing the time base of a signal 
without a significant change in frequency. Another object of the 
present invention is to provide an improved apparatus and method 
for frequency conversion capable of a significant decrease or 
increase of the time base of a signal without significant change 
in perceptible frequency. 

Another object of the present invention is to provide 
an improved apparatus and method for frequency conversion capable 
of use with a computer based storage and retrieval system of 
prerecorded programs or information. A further object of the 
present invention is to increase the reproduction utilization 
capabilities of video and audio recorders, movies and films, 
answering machines, voice mail boxes, and other signal storage 
systems. Other objects and a fuller understanding of the 
invention may be had by referring to the following description 
and claims, taken in conjunction with the accompanying drawings, 
in which: 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a diagram of the preferred embodiment 
of the invention. 

Figure 2 shows a diagram of a prior art MPEG 2 
Development system. 

Figure 3 shows a diagram of the preferred embodiment 
of the User Remote of the invention. 

Figure 4 shows a program recording priority spread 
sheet form of the present invention. 
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Figure 5 shows the spread sheet of Figure 4 with 
typical user entries. 

Figure 6 shows a diagram of the Data Manager and Data 
Storage portions of the invention. 

Figure 7 is a block diagram of the theory of the 
frequency conversion; 

Figure 8 is a block diagram illustrating an improved 
frequency converter system connected to an example input signal 
having a first frequency sustained for a first length of time for 
generating an example output signal having substantially the same 
first frequency sustained for a second length of time; 

Figure 9 is a graph illustrating a single cycle of an 
example input analog signal at a first frequency; 

Figure 10 is a graph illustrating a translation of 
the single cycle of the input analog signal of FIGURE 8 into 
digital form; 

Figure 11 is a graph illustrating the selection of a 
digital sample for the signal of FIGURE 9; 

Figure 12 is a graph illustrating the addition of a 
duplicate of the selected digital sample from the samples of 
FIGURE 10 to provide a sample digital form of a modified output 
signal; 

Figure 13 is a graph illustrating a type of linear 
interpolation of the signals of FIGURE 11; 

Figure 14 is a graph illustrating a transformation of 
the digital samples of FIGURE 12 into an output analog signal; 

Figure 15 is a graph comparing a single cycle of the 
input signal of FIGURE 8 and the output signal of FIGURE 13; 

Figure 16 is a graph setting forth the output signal 
of FIGURE 13 as actually perceived by the consumer due to its 
production at a higher reproduction rate than the input signal of 
FIGURE 8; 
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Figures 17-18 are drawings demonstrating the sampling 
and nature of signals; 

Figures 19-21 are figures like FIGURES 11-13 showing 
an example deletion of a digital sample; 

Figure 22 is a detailed block diagram of a frequency 
converter system; 

Figure 23 is a graph comparing a first signal and a 
second signal and illustrating the reset of the phase angle of 
the signals at a common zero crossover; 

Figure 25 is a figure like FIGURE 23 illustrating a 
reset at the top of a signal waveform; 

Figure 24 is a graph of the constant, non-reset first 
signal in FIGURES 23 and 25; 

Figures 26-27 are circuit diagrams of example frequency 
converter systems; 

Figures 28-29 are block circuit diagrams of a MPEG 
implementation of the invention; 

Figure 30 is a block circuit diagram of a multiple 
channel device; 

Figure 31 is a block diagram of a circuit for signal 

playback; 

Figure 32 is a series of representational block 
diagrams setting forth an example resetting of memory lines in an 
expansion or contraction frequency conversion device; and, 

Figure 33 shows an adaptive filter network for 
incorporation into the system of FIGURE 31. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 
Figure 1 shows a diagram of the preferred embodiment of 
the invention as used in a home or business environment . Figure 
1 includes an example programming source and delivery channel 
which provide program and/ or data material over a delivery 
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channel, a delivery channel interface coupled to' the delivery 
channel and operable to receive said program material and/or data 
therefrom and provide said program material and/or data in a 
stream to a data manager including a program data processor and a 
program data storage, said data manager being coupled to the 
program data storage to store selected portions of the program 
material and/or data in the program data stream, with the data 
manager also preferably coupled to a removable data storage to 
store selected portions of the program material and/or data in 
the program data stream or previously stored program material 
and/or data from the program data storage on a removable media, 
and with the data manager also coupled to a user display via a 
program data processor to retrieve and/or access program material 
and/or data from the program data storage, removable data storage 
and/or program data stream or a combination thereof to provide 
and/or access program material and/or usable data to a user such 
as a user display including a computer, with said user display 
being coupled to a user remote whereby the user may interactively 
program and control the data manager and other components in 
order to select, control and/or utilize the program and/or data 
being accessed. The data manager also is coupled to an alternate 
user display which is coupled to an alternate user remote which 
allows the same and/or other program material to be displayed 
and/or utilized including separately from the first user. 

It will be understood from the teachings herein that 
multiple delivery channels, delivery channel interfaces, program 
data streams, program data processors, user displays, computers, 
and user remotes may be utilized individually or in combinations 
as desired to provide control and/or multiple operations with 
multiple programs or users. For purposes of the present 
disclosure, the following terms are given. the following 
definitions which are somewhat broader than the definitions 
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normally associated with the term. 

Program is the information having some intrinsic 
value which is of use or interest to a person, being or thing. 
Program shall encompass both normal information such as commonly 
known entertainment programs {for example movies and live 
television programs) , informational programs (for example 
commercials and educational classes) and any other information 
including of an audio, video, text, data or natural type which 
may be of value to a particular user. Program includes singular 
information, multiple information, and combination information. 

Programming takes on two meanings: the first is the 
creation, generation, providing or distribution of one or more 
program and/or data streams, the second is the programming of the 
operation of the invention and its usage and/or access of the 
programs. The meaning being used will be apparent from the 
context . 

User is the person, being, thing, facility computer, or 
location which receives and/or uses one or more program. User 
may be referred to as a location, i.e. the User Location. There 
is no requirement that there be only one user, or that a user who 
operates the invention be the same user who utilizes the program. 
There may be multiple simultaneous or sequential users. 
Additionally, the program data stream and/or program data storage 
may be considered a user in that it may automatically operate the 
invention to provide information which the invention utilizes to 
determine the need^ajaa-er priority for storing or displaying a 
program. There may be another user who is a person simultaneously 
watching or utilizing a stored program. 

Viewing is the use of information by the user and shall 
include visual use, auditory use, computer use and/or virtual use 
as will be apparent from the context in which the term is used. 
Viewer is a user which is viewing. 
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Display is the presentation of information for possible 
utilization, for example by viewing (that is seeing and/or 
listening) or processing (that is utilizing at a then non- 
recognizable level) . User Display is the presentation and/or 
utilization of information for possible utilization primarily by 
the User, for example by viewing, which information may also be 
utilized by others. 

Data shall apply to the information of any nature 
carried or stored by the media, channel or interface and will be 
recognized by one of ordinary skill in the art to take on 
different forms including programs as will be apparent from the 
context of the usage . 

Channel is used both in the communications theory sense 
where it applies to the means or mechanism by which data is 
stored, transmitted or delivered, and is also used in a program 
sense such as conventional television channel sense where it 
means a traditional broadcast, cable, satellite or tuner channel, 
which usage will be apparent from the context. 

Key is an actuator or circuit suitable for interface 
with and/or operation by the user. Key Label is an identifier 
associated with a Key and which is preferred to be recognizable 
by the user. 

In Figure 1, programming is provided at a source by any 
of the known methods for actual programs and/or data, such 
methods including optical disk, magnetic disk, film, recordings, 
electronic memory, hard drives, etc. 

For the preferred embodiment, a television video file 
server such as the Hewlett Packard video file server available 
from Hewlett Packard of Palo Alto, California is utilized. It is 
preferred that the program for this example system consist of 
television video, multiple audio and ancillary data including 
closed captioning, time code, presentation time stamp and program 
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schedule information such as those commonly known in the art 
and/or primary xiata^ — it -do proforro d to consist of multiple 
simultaneously provided programs and/or data. The programs of 
this preferred example system are preferably compressed or 
redacted, for example according to the f JA~&EXf~2~ standard, and 
coupled to N multiple delivery channels for satellite or other 
transmission over a large area of land whereby it may be received 
by numerous users, all as commonly known in the art. 

Although such programming services and delivery 
channels are commonly known in the art, it will be suggested 
herein that these commonly known facilities may be modified to 
further accommodate the present invention. For example, at the 
User location, the transmitted programs are received by one or 
more delivery channel interfaces which couple to the N particular 
delivery channels and provide O program data channels, where N 
and O are numbers represent 1 or more and need not be the same. 
It is preferred that all of the associated data for each of the 
programs be provided on a single MPEG 2 compressed data stream 
out of the delivery channel interface from which the data is 
recovered, for example all of the television video, multiple 
audio and ancillary data including closed captioning, time code, 
presentation time stamp and program schedule information for a 
particular program be carried together on a single MPEG 2 format 
program data stream. 

The O program data streams are coupled to a data 
manager having one or more associated program data processors. 
The data manager is preferred to operate to store all of the O 
program data streams in a program and other data storage device 
as the data streams are presented. It is preferred that the data 
storage device have a random access capability such as that 
provided by semiconductor, magnetic or optical storage 
technologies, such as known computer industry RAM, hard disk and 
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optical disk drives. The data storage device could also have 

sufficient speed and memory to capture all incoming programming 

for later individual non-real time processing. It will be 

recognized that there is no requirement as to the data rate or 

regularity or speed of transmission of programs, although it is 

preferred that the Data Manager and Program Data Storage operate 

to receive and store program data as it received. 

It should be pointed out that with a finite amount of 

program data storage capability, sooner or later there will be no 

more storage available for incoming programs. When that event 

occurs or when it is identified that that event is nearing 

occurrence, the Data manager is preferred to be previously 

programmed to start losing or discarding known previously stored 

data or data types according to a known set of priorities. It 

will be noted that the known set of priorities may be changed 

from time to time as will be described below. Upon the 

identification of no more storage being available for incoming 

programs, and the possible resultant loss of previously stored 

data, it is preferred that overwriting occur based on the amount 

of remaining storage and the rate of desired incoming program 

data, based on parameters and priorities which may be selected by 

one or more chosen user. In this respect, it is preferred that 

any multiple viewers are to be given priority such that a higher 

priority user's inputs are performed at the expense of a lesser 

priority user's inputs in the event both can not.be accommodated. 

c , Aooa ca - 

As an example, the user could cjaose to store all of the programs 
from a particular program, network or service (for example a news 
program or data stream) while ignoring, overwriting or not 
storing another particular program, network or service (for 
example a court room television channel) . As an additional 
feature, the user may direct the Data Manager to keep memory 
available for a particularly desirable program, network or 
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service, and/or to only store a given quantity of one or more 
program network or service. For example, the user may direct the 
Data Manager to only store the last three days worth of 
channel A, the last four programs of program B, none of program 
C, none of channel D and to always store programs of type E. 
Quantities may be chosen in units of time, programs, elements of 
a series or according to program, channel, network or service 
content or combinations thereof. It will be recognized that such 
complexities tend to create exponentially growing requirements on 
the Data Manager with each additional parameter, such capability 
is still well within the abilities of current programming and 
computer operation capabilities of those of ordinary skill in the 
art. Additionally, it will be recognized that a Removable Data 
Storage capability {in addition or separately) may be required or 
desirable with the user having the ability to direct the Data 
Manager to cause a known program or programs to be stored thereon 
either from the Program Data Storage, or directly from the 
Delivery Channel Interface. The Removable Media may then be 
physically removed and/or replaced - for example for storage for 
archive purposes or to be moved to other locations . 

The selection of such program or programs may also be 
subject to the same or different selection requirements and 
priorities, as the program data which is stored on the Program 
Data Storage . Program data from removable Data Storage Media may 
also be read therefrom for example via media previously archived 
or received from other locations . The Data Manager & Program Data 
Processor (s) also couple to one or more User Display which is 
preferred to be a video CRT, LCD or Active Matrix type. display 
and associated audio transducer such as a speaker set or 
headphones. Each User Display also couples to a User Remote which 
is preferred to be a keypad or keyboard which is preferably 
wirelessly bidirectionally coupled thereto, such as by infrared 
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beams. It is preferred that the User Remote additionally have the 
capability to program and reprogram the keys thereon, in order 
that a small number of keys may perform multiple functions, 
and/or in order that the user may chose convenient key labels for 
the keys which facilitates the viewers identification and use 
thereof. 

It will be recognized that it will also be useful for 
the User Remote to contain a message display, for example such as 
an LCD readout, beeper or speaker, which allows the user to 
receive messages directly or indirectly from other parts of the 
system without having to receive them from the User Display. 
This is especially useful for Human viewers where an audible 
signal would aid in locating the remote or signaling invalid 
operation, and an LCD display would allow messages to be 
displayed during programming type operations without requiring 
the user to look away from the remote. It is desired that the 
Data Manager also include one or more Program Data Processor 
which function to convert the MPEG 2 format program data stream 
from the Delivery Channel Interface, Program Data Storage and/or 
Removable Data Storage into a format which may be displayed on 
the User Display. In addition, it is preferred that the Program 
Data Processor provide capabilities for displaying multiple 
programs on the User Display, such as can be provided by 
displaying different programs in multiple windows on a CRT type 
video display, and for providing special effects such as spatial 
manipulations or enhancement of images or of sounds and quick 
viewing and time compression (or expansion) capability, for 
example such as that described in copending U.S. Patent 
Application serial number 089,904. Also, it is seen in Figure 1 
that the Data Manager may be coupled to Other Services, for 
example such as telephone wire, fiber and other terrestrial 
communications/ remote video cameras, alarms and actuators, such 
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as baby room monitors and doorbells. 

Communications with the Programming Services which 
operate the Programming Sources to provide the Programs - over the 
Delivery channels is an important capability provided by the 
Other Services. In this fashion account charges and payments may 
be made at the Users wish, for example to automatically arrange 
for payment for pay per view or other special programs. 

Of special importance is the ability to interface with 
program guide services which provide information on upcoming 
programs. In this fashion, the Data Manager may display lists or 
charts of upcoming programs along with other program information 
thereby allowing the User to select wanted ones of the upcoming 
programs for storage. It will also be noted that many times such 
upcoming program information is carried as embedded information 
on the Program Data Streams themselves, thus the Data Manager may 
acquire this information directly from the data stream. 

The components used to practice the invention of Figure 
1 are individually available to one of ordinary skill in the art. 
The components may be procured and interconnected by one of 
ordinary skill without undue experimentation or further invention 
by resorting to the teachings herein taken in accordance with the 
prior art. For example, the Programming Source may be provided by 
the aforementioned Hewlett Packard video file server and the 
DiviCom MPEG- 2 Encoder, MPEG- 2 ReMultiplexer operating with the 
DiviCom MPEG- 2 System Controller available from DiviCom, Inc. of 
Milpitas, California. The Delivery Channel Interface may be 
provided with one or more standard receiving devices well known 
in the industry, for example a standard satellite DSS tuner such 
as manufactured by Thompson under the RCA brand name in 
Indianapolis IN. Data storage by the Program Data Storage and 
Removable Data Storage is preferred to be provided in a single 
unit which operates with removable writable optical discs. These 
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functions may be separated however if desired, for example to 
utilize lower cost storage such as computer hard disks or faster 
semiconductor memory for the Program Data Storage. The Program 
Data Storage may be provided for analog or Dl video and audio by 
the RAMSES video ram recorder available from EVS Broadcast 
Equipment of Lie'ge Belgium or by the DLC-V500 available from 
Pioneer of Long Beach CA. The DLC-V500 utilizes removable optical 
discs for storage and may also provide the Removable Data Storage 
either as a separate unit or in combination with the Program Data 
Storage combined in a single unit. Both the Program Data Storage 
and the Removable Data Storage may also be implemented for 
storing the MPEG-2 compressed program data stream with standard 
computer storage technology, computer hard drives and optical 
WORM drives respectively. Decoding of the MPEG-2 data stream from 
any of the sources may be performed with the DiviCom companion 
consumer, commercial or professional decoders, and provide stan- 
dard analog or digital video signal (s) for use by the User 
Display or the RAMSES recorder. The technology for the User 

Display and infrared technology for communications with the User 

+• 

Remote are commonly available in consumer television sets. The 

*- 

Data Manager and communications with the User Display may be 
implemented with a personal computer such as any of the common 
multimedia Pentium based computers or a VME bus computer such as 
the Sun Sparc. Video interface and Digital Signal Processing 
boards which may be used to implement the Program Data Processor 
are available from numerous DSP companies including Atlanta 
Signal Processors, Inc. of Atlanta GA. , Ariel Corporation of 
Highland Park, NJ, Analogic Corporation of Peabody, MA, and 
Traquair Data Systems, Inc. of Ithaca, NY. Interface cards to 
interface the computer to the other services are readily 
available, for example the IBM Windsurfer Communications Adapter 
from IBM Corporation, Research Triangle Park, NC. 
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Figure 2 shows a prior art MPEG-2 development system 
which may be used by those who wish to implement and practice the 
invention on existing hardware with suitable modifications. The 
C-Cube MPEG 2 Development system available from C-Cube 
Microsystems of Milpitas Ca. provides a Sun Microsystems Sparc 
plug in card (VME bus) which provides analog or Dl digital video - 
input, analog video out, stereo analog audio in and out, control 
of external devices (such as an HP video file server or EVS 
RAMSES) via RS232 port, user keyboard, full color monitor, 
storage of MPEG encoded Video on both hard drive and removable 
tape drive via SCSI bus, Ethernet communications and serial MPEG 
data communications. To practice the invention, one adds the 
necessary programming, wireless communication and the User 
Remote, which may be achieved via the RS232 serial port or 
Ethernet, and to connect an MPEG program data stream from the 
Delivery Channel Interface via the serial MPEG data port. Similar 
development systems are available from other manufacturers. 
Programming of the hardware operation is provided via the 
keyboard and will be well within the capabilities of one of 
ordinary skill in the art from the teaching herein. 

Figure 3 shows a diagram of the preferred embodiment of 
the User Remote of the invention. While the User Remote is shown 
in its preferred embodiment form for use by a person speaking 
English, it will be understood that it may be modified as 
appropriate to accommodate persons speaking other languages, or 
with various handicaps, or to operate with mechanical, electronic 
or optical interfaces, the latter being particularly well suited 
for interfacing with other devices. The preferred User Remote of 
Figure 3 contains an infrared transmitter and infrared receiver 
for communications with the User Display or other components of 
the invention by way of infrared light beams, an LCD display and 
video imager for visual interaction with the user, the video 
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imager for imaging scenes or materials which the user points the 
imager to and the LCD display for displaying text messages, 
graphics or images to the user, programmable keys A, B, C, and D 
each having a programmable label A, B, C, and D, respectively, an 
audio transducer consisting of a speaker and microphone for audio 
interaction with the user, up and down keys /for communicating 
the users desire to increase or decrease a particular 
function; oonf i^gurabl o ^f unction keys PI, F2, F3 and F4 with 
associated configurable labels, and a standard 12 key numeric 
keypad. The standard 12 key numeric keypad is preferred to be 
fixed in programming and configuration, and to have none hanging 
labels, however if it is desired to facilitate more extensive 
capabilities the keypad may also be programmable or configurable, 
and have associated changeable labels. 

In operation, the user stimulates or actuates the 
various keys and sensors in proper sequence and combination in 
order to direct the Data Manager and other components of the 
system to perform in a fashion to present to the user the desired 
program material and/or access desired information on the User 
Display or User Remote. It will be recognized that the Data 
Manager is capable of communicating with the User Remote to both 
configure the programmable keys A-D and program their associated 
labels accordingly, as well to display any desirable messages on 
the LCD display or couple any desirable sound to the audio 
transducer. In this fashion the User Remote may function to 
operate, via the User Display and Data Manager, all needed system 
functions. It is also recognized that the User Remote may 
operate directly with other system components to perform wanted 
functions. As an example of the preferred operation of the User 
Remote, in the present invention, in normal operation key A is 
labeled sound, and when the operator actuates the A key the 
message "UP FOR LOUDER -DN FOR SOFTER" appears on the LCD display. 
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By actuating the UP or DN key the operator controls the sound 
level. Similarly, the B key is used to change the channel up or 
down, the C key is used to speed up or slow down the playback 
speed of previously stored programs, and the D key is used at: the 
first actuation to temporarily stop the playback of the program 
(this operates with program from the data stream or previous 
storage as will be discussed below) and at the second actuation 
to continue the program from where it stopped. Similarly, the 
programmable function keys normally operate with Fl being used to 
initiate or answer phone (or video phone) calls, much like 
lifting the handset on a standard telephone, F2 to replay 
previously stored phone messages much like pressing the play 
button on a standard answering machine, F3 to select programmable 
attribute menus for configuring the operation of the system and 
F4 for selecting programmable display functions of the User 
Display. Function key F4 operates to inform the Data Manager to 
display graph functions for COLOR, HUE, BRIGHTNESS and CONTRAST 
on the User Display, or alternatively on the User Remote LCD. In 
addition, keys A-D and their labels are reconfigured to 
indicate COLOR, HUE, BRIGHTNESS and CONTRAST, with adjustment of 
the particular function desired performed by actuating the 
associated key followed by the UP. or DN key. A second press of 
the F4 key operates to inform the Data Manager to display graph 
functions for frequency equalization of the audio reproduction on 
the User Display, or alternatively on the User Remote LCD. In 
addition, keys A-D and their labels are reconfigured to indicate 
they are in use for the functions of left to right and front to 
back balance of the sound channels. Adjustment of the frequency 
response is performed by actuating the numeric key associated 
with the desired frequency to be adjusted (as indicated on the 
graph display) , followed by the UP or DN key. 

Function key F3 operates to inform the Data Manager to 
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display the first level menu selections on the User Display or 
User Remote, and to reconfigure appropriate programmable keys and 
associated labels accordingly. It may be noted that it is 
preferred that one of the menu selections will be to display 
menus on the User Display or User Remote LCD or both. After all 
key actuations, it is preferred that a timer be started which 
will return the User Display and User Remote to the previous 
operation 10 seconds after the last key press, in order to resume 
normal displays. Alternatively, one or two of the programmable 
keys may be programmed to perform CANCEL or END functions. 

Figure 4 shows the spread sheet of the preferred first 
menu available under the F3 key, this menu allows the operator to 
select individual incoming program data streams to be stored or 
ignored and their method of display. For purposes of explaining 
the instant invention it will be assumed that there are four 
incoming data streams corresponding to four programming services 
or networks which provide SPORTS. HOBBIES NEWS and MUSIC. Other 
data streams could also be utilized. 

The User Display or User Remote LCD lists a spread 
sheet type display with the left most entry being the Program 
Data Stream Number with vertical columns for IDENTIFIER, PROGRAM 
STORAGE TIME 1, STORE PERIOD, MOST/LEAST RECENT OVERWRITE, 
STORAGE TIME 1 PRIORITY, PROGRAM STORAGE TIME 2, STORAGE TIME 2 
PRIORITY, STORE PERIOD, MOST/LEAST RECENT OVERWRITE. If desired, 
additional entries may be provided for. 

The IDENTIFIER column is used by the user to enter a 
unique identifier made up of symbols, alpha or number characters 
or both by which the program data stream is to be known by the 
user, for example SPORTS for Data Stream 1, HOBBIES for Data 
Stream 2, etc. Such capabilities are described in considerable 
detail by Beery in U.S. Patent 5,068,734, which prior art patent 
is incorporated herein by reference. The characters, symbols and 
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other information for entry in the spread sheet may be selected 
from a list at the bottom of the spread sheet display by use of 
the programmable keys A-D which are programmed and labeled to act 
as cursor movement keys . 

The function key Fl is configured to act as a select 
key to first select the spread sheet location into which 
information is being programmed and then select the character, 
symbol, etc. on which the cursor is placed. Information so 
entered via the User Remote is transmitted to the Data Manager 
which records and stores the information and uses it to 
facilitate User selection of programs to be watched as described 
by Beery. 

The PROGRAM STORAGE TIME 1 column is used to enter a 
range of times during which a storage priority is to be assigned 
to the corresponding program data stream. For example, if the 
User wishes to record a sports highlights show which is 
transmitted from 6:00 to 7:00 PM Monday thru Friday, this 
information is entered via cursor movement as previously de- 
scribed. 

The STORAGE TIME 1 PRIORITY is a numerical priority 
which is used to determine what programs are overwritten in the 
event of insufficient memory. The lower the priority number the 
more desired the storage of the program. Thus if it is desired to 
store two programs at * the same time, and memory is only available 
for one, the higher priority (lower number) program is stored. 
In addition, if insufficient memory is available and there is a 
lower priority program already stored in memory .the higher 
priority program will overwrite it. The STORE PERIOD lets the 
User assign a length of time the program is stored before is may 
be overwritten by a lower priority program. 

The MOST/LEAST RECENT OVERWRITE lets the User choose to 
have the incoming program overwrite the most recent or least 
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recent stored version of the same program if there is no memory 
available to store an incoming program because of relatively low 
priority as compared to what is stored in memory. 

A second set of storage parameters is also preferably 
provided in order to allow more storage selections. If desired, 
even more storage parameter sets may be provided. After filling 
in the desired information, a typical spread sheet might look as 
shown in Figure 5. Note that there are two #2 priorities in these 
entries. This is acceptable because they take place at different 
times. In the event the User attempts to enter conflicting 
information which the Data Manager would be unable to 
accommodate, a warning is preferably provided, such as by causing 
a short beep to be emitted from the audio transducer and flashing 
the conflicting entries. After finishing the entries of 
information in the spread sheet, the user exits that feature. 

The F3 key shown also provides access to a listing of 
all currently stored programs in memory which the User may use to 
play back a stored program. The user will simply highlight the 
stored program he wishes to view with the cursor arrow keys 
(programmed key A-D) and again presses the F3 key. An exit option 
is provided and may be highlighted by the cursor with the F3 Key 
to exit. The F3 key further provides access to system default 
conditions. These defaults include the following: Allow 
overwrite of program after viewing entire program and allow 
overwrite of program minutes after viewing etc. These functions 
also determine how long to wait before a viewed program is 
discarded. For example it may be desired to retain educational 
programs, but to immedia€ely discard entertainment programs. It 
further may be desired to allow access to specific programs or 
types by password. This prevents unauthorized persons from 
viewing identified programs. Playback speed is also setable. 
This feature allows stored programs to be played back faster (or 
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slower) than normal, thus coupling the information or 
entertainment delivery to the viewers ability to receive and 
process . One can also select program types for storage and/or 
preferred viewing programs and program types. These are a user 
convenience features . Configuration to interface with other 
services allows the hardware to match the particular set of 
services the user desires. The Auto Store Buffer reserves time 
and overwrite priority. This allows a User to pause a program 
being received from the Program Data Stream with the invention 
operating to continue storing the received data so that when the 
User returns to viewing and disables the pause the User begins 
where he stopped without loss of program material . The buffer 
reserve time determines how much memory is reserved for this 
storage function and consequently how long the User may keep the 
system in pause and be guaranteed of no loss of program. The 
Auto Store Buffer catch up rate allows a user to return to 
viewing a program being received from the Program Data Stream 
after a pause, while insuring he will be viewing the stored 
portion of the program from the Auto Store Buffer. It is 
desirable to read this information out of the buffer at faster 
than normal rate. In this fashion, the program is read out of the 
buffer faster than the program being written into the buffer and 
consequently the buffer length will be continually shortened to 
zero. This feature both returns the User to real time viewing, 
thus allowing him to finish viewing the program when it actually 
ends so he can go on to watch another Program Data Stream program 
when it starts,. and returns the Auto Store Buffer to an empty 
state in preparation for pausing at another interruption. It 
might be noted that when the stored portion of the program from 
the Auto Store Buffer is read out of the buffer at faster than 
normal rate, or when stored programs are read out at faster or 
slower than the normal rate, that an audio pitch shift and video 
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motion artifact would normally occur. It is preferred to correct 
these artifacts by use of the invention described in copending 
U.S. Patent Application serial number 322,069 [incorporated 
herein by reference] . 

The present invention includes a conversion system as 
set forth in 322,069 and devices that incorporate it, which 
conversion system can convert an input signal having frequency 
related information normally sustained over a first length of 
time into an output signal having substantially the same 
perceived frequency related information, with the information now 
normally sustained over a second length of time alternately just 
frequency, and/or frequency and length of time can be modified. 
The theory behind this operation is , shown in the FIGURES, 
including FIGURE 7. The theory behind the invention involves 
getting an input signal 10 (Block I) . This signal has frequency 
based information sustained over a period of time. This input 
signal 10 is provided to a signal modification circuit 50 (Block 

II) . The signal modification circuit 50 adds or subtracts 
samples 15 to or from the input signal 10 according to certain 
principles, mathematical principles normally based primarily on 
the ratio of frequency and/or time between the input 10 and 
output 100 signals and the complexity of the signals. The signal 
modification circuit 50 then remits an output signal 100 (Block 

III) , which output signal 100 has a relationship to the input 
signal 10 as set by the certain mathematical principles. One 
skilled in the art should recognize that the devices disclosed in 
this application could alter frequency over the same length of 
time, alter frequency and length of time, and otherwise function. 
The easiest way to do this would be by altering sample and/or 
clock rates. For uniformity, this application will primarily 
utilize as an example devices producing an output signal that 
perceptibly has the same frequency related information as the 
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input signal 10 and may also be sustained over a different length 
of time. 

In this operation, both the input 10 and output 100 
signals have frequency related information on them. The output 
100 signal can be either expanded or compressed relative to the 
input signal 10. The signals themselves can be audio, 
television, computer signals, or other signals having frequency 
related informat ion thereon. Further, the devices can be used in! 
singular form (for example a television video signal) , paired 
form (for example right and left stereo audio signals) , or in 
other combinations including synchronizing the output signal to a 
related signal (for example synchronizing audio to video) . The 
signals themselves can be in analog or digital form. A digital 
form is presently preferred in that technology is presently more 
established for digital processing of complex wave forms. 
However, with the increasing advances in analog circuitry 
including the use of charged coupled devices (CCD's), it is 
envisioned that soon analog processors may be able to process the 
complex signals as well and perhaps better. 

The digital signals may be coded in pulse code 
modulation (PCM) , pulse width modulation (PWM) , pulse length 
modulation (PLM) , pulse density modulation (PDM) , pulse amplitude 
modulation (PAM) , pulse position modulation (PPM) , pulse number 
modulation (PNM) , pulse frequency modulation (PFM) , pulse 
interval modulation (PIM) , or other coding scheme. Pulse 
amplitude modulation will be utilized in the explanation of the 
invention. The location of the signal modification circuit 50 in 
the overall replication path is not critical. In most instances, 
the signal modification circuit 50 would be located after some 
sort of signal storage means for modification of the stored 
signal. This is generally preferred in that the stored signal 
would contain the highest quality signal. Such stored signal 
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could also be otherwise used. However, the signal modification 
circuit 50 could be located prior to the storage means or even 
within such storage means. The circuit 50 could also operate in 
real time. Further, the order of the conversion steps are not 
critical as long as all steps are accomplished. For example, the 
clocking shift and analog to digital conversion could occur prior 
to real time signal modification in the overall frequency- 
conversion of an analog signal. An example of this would be 
playing an answering machine at high speeds with subsequent real 
time frequency conversion to lower the voice pitch to normal 
values . 

Further example in FIGURE 7 the storage means could be 
located before/after or between any of the blocks of circuitry at 
points A-H respectively. The operation of the invention is thus 
also not dependent on a storage location. FIGURE 8 is a block 
diagram of the signal modification circuit 50 receiving an input 
signal 10 sustained over a first length of time 13 at a first 
ascertainable frequency. In real time this length of time 13 
would be the period of production of the input signal 10. As the 
signal 10 utilized as a uniform example in this specification is 
an analog signal, a digital converter 14 converts the input . 
analog signal 10 into a digitally sampled version 20 of the input 
analog signal 10 (if the input signal 10 was itself digital or 
already a digitally sampled version of an analog signal, no 
conversion would normally be necessary; oversampling however, 
might be appropriate. The inclusion of the converter 14 in the 
modification circuit 50 is thus dependent on the nature of the 
processed signals) . In the particular circuitry example of the 
figures, the input signal 10 is an analog signal having an alpha 
length 13. Here, applicant defines alpha length as the time 
duration of a contiguous signal block, exclusive of any reset 
operations (reset operations will be addressed in detail below) . 
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This input signal 10 is normally replicated over a certain time 
period, a period normally directly related to the alpha length. 
The input signal 10 normally exists for reproduction over a 
certain set length of time, a time length analogous to inverse 
clock rate including real time. In FIGURES 8 and 22, input clock 
refers to the speed of production or reproduction of the input 
signal . The input sample rate refers to the rate at which 
discrete time samples are presented to the signal modification 
circuit . The input clock and input sample rate may or may not be 
related. For example, if the input source is an analog tape 
player, the input clock would refer to the speed of the tape. 
Tape speed might be variable, while the input sample rate may or 
may not be variable. As another example, the input source may be 
a compact disk player outputting digital samples at a 44.1kHz 
rate. In this case, no continuous time to discrete time 
conversion is necessary. If no sample rate conversion was used, 
the input sample rate would here be the same as the input clock 
rate. If the 'speed of playback of the compact disk was varied, 
then both the input clock and input sample rate would vary. The 
output sample rate is the rate at which discrete time samples are 
output from the signal modification circuit. It. may or may not 
be equal to the input sample rate . The output clock rate refers 
to the speed of production of the output signal, and may or may 
not be related to the output sample rate. The input signal 10 is 
normally preferably fed into a digital converter. 14 in order to 
replicate such input signal 10 in digital samples 15. The nature 
and rate of the digital sampling is selected in accord with the 
overall circuitry design. Examples of the type of digital 
sampling that can be utilized have been previously set forth. 
For uniformity, the preferred embodiment of the invention will be 
set forth with pulse amplitude modulation (PAM) digital sampling. 

It is preferred that the digital coding and/or rate be 
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selected in respect to the nature and frequencies of both the 
input and output signals. For example, according to the sampling 
theory, a sampling rate of a little over twice the highest 
expected frequency will allow for the accurate reproduction of an 
analog signal with minimal distortion. An example of this is the 
44.1 kHz sampling rate for common compact disks. In addition to 
this, the sampling rate must be selected in order to provide for 
the compress ion/ expansion of the signal in an accurate manner. 
This entails a review of the signal content. In specific, if a 
computer on/off binary signal was involved with a conversion of 
3:2, a sampling rate three times the clock speed of the input 
signal would provide for completely accurate conversion (fig 11) . 
However, with an audio signal at the same somewhat extreme 
example 3:2 reduction, a sampling rate of twice the frequency of 
the input audio signal (for example a sampling rate of 44.1 kHz) 
would provide a normally unacceptable result due to the 
distortion on the output signal 100. The reason for this would 
be that aliasing would occur if one third (1/3) the samples were 
removed. It is therefore necessary to sample the audio input 
signal 10 at a rate much higher than the Nyquist rate in order to 
provide for an acceptable output signal for the analog signal 
(fig 17) . Over and above this restriction, it is preferred that 
any input signal 10 be sampled at as high a rate as possible, in 
order that the addition/deletion of individual samples would have 
a minimal effect on the information available on such input 
signal 10. For example, the deletion of one out of every ten 
samples at a 10,000 times over sampling rate would have less 
artifacts than the deletion of one out of ten samples at a ten 
times over sampling rate although both provide the same 10 percent 
(10%) signal compression. The reason for this is that with 
higher rate sampling, the many artifacts which would be produced 
would occur at an extremely high frequency, with many occurring 
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at a frequency above that perceptible to the senses of the 
consumer. The Philip's pulse amplitude modulation at a standard 
rate of 256 over sampling (256x44.1 kHz) is a natural sampling 
technique for the invention in audio applications. 

The difference ratio 51 that is input to the actual 
modification circuit 52 determines the scope and nature of the 
relationship between the input 10 and output 100 signals. The 
general concept is that there is an input signal 10 which has 
frequency related information, which input signal 10 further has 
some frequency and/or time ratio to the output signal 10 0, 
normally a ratio based on the times of expected signal 
production. if time is the determinant, the difference ratio is 
selected such that the output signal 100 when perceived has the 
same frequency related content as the input signal 10 . 
Alternately the output signal 100 may have the same time as the 
input, but a different frequency or both may be varied 
simultaneously . 

The difference ratio may be defined as the output 
frequency time product, divided by the input frequency time 
product. For example, suppose that the difference ratio is 
0.855. If the input and output times are the same, then the 
output frequency is 0.855 the input frequency. If the input and 
output frequencies are the same, then the output time is 0.855 
the input time. If the output frequency is 0.95 the input 
frequency, then the output time would be 0.9 the input. time, 
since 0.95 multiplied by 0 . 9 equals 0.855. 

The difference ratio 51 can be set manually or 
automatically. An example of the former would be having a 
technician dial in a factor representative of the input length 
and then a second factor representative of the output length. 
This type of manual setting would be particularly appropriate 
where the technician knew that a thirty (30) minute television 
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program needed to be inserted into a twenty eight (28) minute 
time slot. As an example of the automatic setting, in television 
signals the horizontal sync pulses could be utilized to 
automatically decompress a tape recorded television movie. This 
type of automatic functioning would be particularly appropriate 
for signals having known, repetitive, determinable attributes or 
where the function of the circuitry can be readily determined 
(for example a profanity dump) and/or a known real time length 
and known time for actual presentation. 

In the circuitry of FIGURES 8 and 22, the difference 
ratio is as previously defined. This ratio has been previously 
determined. Pitch shift may be obtained by sample insertion or 
deletion, .if the input and output sample rates are the same. 
Alternatively, pitch shift may be obtained by using differing 
input and output sample rates, without sample insertion or 
deletion. Additionally, a combination of sample 
insertion/deletion and differing sample rates may be used. The 
sample rates preferably are at least greater than the Nyquist 
rate for both input and output. Over and above this, distortion 
considerations could require that the input signal be sampled at 
a rate much higher (for example 20 times) the highest input 
frequency in order to insure production of the output signal, with 
minimal distortion. Note, however, that in noncritical 
applications the sampling rates can be much lower, particularly 
if the signals can be^ka nd width limited while retaining 
acceptable information content (an example of this would be Jaaaeh 
^i^tti limiting an audio signal to 5kHz) . 

As shown in FIGURES 9-23, the modification circuit 52 
selects at least one sample 22 from the digital sampled version 
20 of the input signal 10 and generates a second plurality of 
digital samples 120 by altering the number of the digital sampled 
version 20 of the input signal 10 by the selected digital 
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sample (s) 22. This can be by addition to expand (samples 122 in 
figs 12 and 13) or by subtraction to compress (samples 27 in fig 
19) as appropriate. The location of the added/deleted samples is 
selected in view of the signal content so as to minimize 
artifacts. For very high oversampling rates, the samples can be 
spread out over the entire alpha length of the signal. For lower 
oversampling rates, locations of least slope, least differences, 
signal peaks, or other minimal signal information points are 
preferred. 

Note that figures 9-21 are given by way of example. 
Other sampling/modification methods could also be utilized with 
the invention. Note also that for clarity of explanation in 
these figures that the input sample is converted to digital by a 
leading edge sample and hold circuit (left edge) , while the 
output sample is converted to analog by a trailing edge 
conversion circuit (right edge) . Alternate conversion circuits 
could be utilized if desired. For ease of comprehension, no 
interpolation is used in figures 9-21. 

The second plurality of digital samples 120 can be 
interpolated to reduce distortion caused by replication or 
deletion of the selected digital sample 22 if appropriate. A 
second digital converter 114 then generates an output signal 100 
from the second plurality of digital samples 120 over the second 
duration of. time 113 (again the inclusion of this convertor is 
dependent on the nature of the output signal) . In the example 
shown, this produces a signal having substantially the same first 
frequency when clocked or reproduced at the new speed provided 
that the rate of occurrence of altered samples 22 relative to the 
input sample rate corresponds to the ratio between the first 13 
and second 113 duration times. Compare the analog input signal 
10 of FIGURE 9 with the output analog signal 100 of FIGURE 16, 
which output signal 100 is being produced during a different, 
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shorter, length of time: In real time 80, the perceived 
frequencies or alpha length of the signals are the same. In this 
respect, it is noted that minor unobjectionable shifts could be 
accepted by the overseeing technician, this even though the pitch 
of the resultant signal is not absolutely accurate. Alternately 
the output may have a different frequency and same duration or a 
combination of different frequency and duration.. 

It should again be appreciated by those skilled in the 
art that either the digital converter 14 or the digital converter 
114 may be optional in the event that either the input signal 10 
or the output signal 100 is a digital signal of suitable signal 
content to allow for acceptable modification. Otherwise, 
oversampling would normally be appropriate. 

The selected digital sample (s) 22 is added (fig 11) to 
the sampled version 20 of the input signal 10 to provide pitch 
correction to facilitate the prior or subsequent decrease of the 
duration time. The selected digital sample (s) 22 is removed (fig 
19) from the sampled version 20 of the input signal 10 to provide 
pitch correction to facilitate the prior or subsequent increase 
of the duration time. This will be discussed more extensively 
later, especially with respect to FIGURES 22-24. 

The signal modification circuit 50 adds or subtracts to 
the apparent alpha length of the input signal 10 according to the 
difference ratio in order to produce the output signal 100. 

With present digital technology, this signal 
modification circuit 50 would begin with digital signals or a 
digital sampling replication 15 of an analog signal. This signal 
modification circuit 50 then repeats (to add or expand) or 
deletes (to subtract or compress) from these samples in order to 
alter the input signal 10 to an output signal 100 in accord with 
the set difference ratio. (Note again that a digital signal 
might, like an analog signal, have to be digitally over sampled 
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to achieve acceptable distortion performance.) 

A more sophisticated signal modification circuit 50 
could average or linearly or otherwise interpolate the modified 
samples in order to optimize the functioning of the device in 
certain applications. 



resulting from applying a 5.6 percent expansion to a sine wave 
was performed. The sine wave period was 36 samples. The purpose 
of the analysis was to determine the relative distortion levels 
resulting from the insertion of samples at different phases of 
the input sine wave. Two samples per period were inserted. The 
type of linear interpolation used in the preferred embodiment is 
essentially the second order approximation method used for sample 
rate conversion. Proakis, John G. , Rader, Charles M. , Ling, 
Fuyun, and Nikias, Chrysostomos L. , Advanced Digital Signal 
Processing, Macmillen, 1992. This particular interpolation 
operates along the entire signal, and weighs two adjacent samples 
in proportion to the distance from the last sample insertion. 
This type of interpolation gives exact (within quantification 
limits) output in the case of a linear ramp signal with constant 
slope (zero second difference) . Three insertion points were 
examined: peaks, zero crossing, and 306 lagging from zero 



crossing. Insertion of samples at the zero crossing resulted in 
the lowest distortion of the three. Insertion of samples at the 
peaks was slightly inferior to the zero crossing. Insertion of 



samples at the 3Qo points resulted in the highest distortion. A 



feature of the zero crossing point of a sine wave is that it is 
the location of minimum second difference magnitude. A feature 
of the positive and negative peaks of a sine wave is that they 
are the location of minimum slope, or minimum first difference, 
magnitude. A feature of points on the sine wave which are 



removed from the peaks or zero crossings, such as the 3Q/b phase 



For example, a theoretical analysis of the spectrum 
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point, is that neither the first or second difference is 
minimized. 

Sample insertion at the peaks and at the zero crossings 
was also investigated for the case of no interpolation. In this 
case, insertion at the peaks gave lower distortion. Distortion 
performance in both cases was significantly worse than that 
obtained where interpolation was used. 

It is envisioned that other interpolation algorithms 
may be advantageous from a performance standpoint . Examples of 
other algorithms would be polyphase subf iltering, higher order 
Lagrange polynomial interpolation, and finite impulse response 
low-pass filtering. In the case of expansion, it may also be 
advantageous for the added sample to be of some value other than 
the value of the immediately preceding sample . 

Customarily, the signal modification circuit 50 would 
contain a delay length preferably of the variable type, the 
length of which must at least allow for the appropriate shifting 
of the signals to add or subtract whole cycles. Specifically, 
the maximum shift between the input signal 10 and the output 
signal 100 should be within the effective delay length of the 
available memory. Further, to provide for a smooth output signal 
100, the present invention preferably uses a memory delay longer 
than this period in order to provide for a seamless operation. 

The present invention accomplishes this with less than 
the amount of memory otherwise needed by comparing a first and 
second signal in order to reset the information in the memory (as 
later described) to add or delete blocks or cycles thus to 
provide for a seamless integration of the signals. In general, 
the more memory that is available, the more time can pass before 
the device is reset subject to an ascertainable artifact 
override. For complex audio signals, up to a point, the quicker 
the resetting; normally the less ascertainable the artifacts as 
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will be described in more detail below. In addition, an even 
greater memory would allow an operator to delete or add blocks, 
cycles, or multiple cycles of signal information with no pitch 
change . 

The delay could be a single memory if variable taps 
were available and the signal was actively followed through such 
variable taps by the signal modification circuit 50. Changing 
over between effectively two separate memory delay lengths 56, 57 
is preferred, which delay lengths are each long enough to provide 
for the later described resetting, because of the way the design 
evolved. A single memory delay element was originally used. 
Later, a second memory delay element was added, since this was, 
at the time/ the easiest way to accommodate the needed later 
described cross fade reset operation. At any given time, 
exclusive of reset operations, one memory circuit would be 
actively utilized in real time by the signal modification 
circuit, while the other memory preferably would continue to be 
updated with input signal data. An example of this is the two 
stretch cell embodiments set forth in FIGURES 26 and 27. 

The circuitry of FIGURES 23 and 27-28 include two 
effective memory lines 56 and 57 together with two modification 
circuits 60 and 61. FIGURE 23 utilizes two separate RAM memories 
while FIGURES 26 and 27 utilize a single RAM memory, with two 
different address spaces allocated respectively to two stretch 
cells to allow resetting. 

The memory lines 56, 57 are preferably RAM memory 
circuits. These circuits provide for a delay necessary for the 
processing to occur. The length of these memory circuits is 
chosen in order to optimize the performance of the overall 
circuitry while at the same time preferably minimizing expense. 
The selection of length is normally a compromise between 
excessive delay versus the ascertainable artifacts which might 
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occur during resetting at an earlier than appropriate time or 
resetting at too high a rate. In general, changing over more 
quickly reduces information loss. However, changing over too 
quickly results in unacceptable artifacts, so a compromise is 
chosen. Also, in general, optimization of performance is 
determinative of the preferable length for the memories 56, 57, 
even though theoretically an infinite memory length could be 
used. {Too long a delay can produce noticeable gap and/or echo 
effects. It also violates EIA/TIA-250-C. ) Due to the use of two 
memory lines, and the changing therebetween (as later described) , 
memory lengths of from 38 mS to 149 mS are more than sufficient 
for a normal audio signal . This memory is sufficiently long to 
allow for the smooth crossover between memories 56, 57 while at 
the same time storing sufficient samples so as to support the 
typically experienced maximum time between crossovers. Note that 
the length of the memory is a design choice. For example, there 
are frequent times of audio silence in a television talk show. 
If the memory was sufficiently long as a function of the expected 
length between silences in audio content, and the duration of the 
silence was at least as long as the section of memory to be 
traversed by the reset, the memory could be reset during these 
times of silence, thus not dumping or repeating any audio 
information. The length of memory required is the expected 
length between silences multiplied by the conversion factor. For 
example, for 0.4 seconds between silences at ten percent 
expansion or compression, 40 mS of memory is typically required. 
Further example in a symphony orchestra a memory technically long 
enough to work may produce artifacts due to too frequent resets. 
A longer memory accompanied by a reset prevention control would 
therefore be used, preferably delaying resetting to a time of 
silence or otherwise when least artifacts would occur. An 
additional example would be with television video content wherein 
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there are frequent times of static images and/or screen changes. 
These can be located with a motion detector. By adding or 
deleting frames thus changing over video during these times, 
video artifacts are reduced to a minimum. Further, if the 
signals of any nature are compressed or expanded prior to signal 
modification and the sampling rate is high enough, and acceptable 
reset opportunities occur with sufficient frequency, conversion 
can occur with low average throughput delay. It is, therefore, 
important to recognize that the invention can be optimized for a 
given signal by slightly altering its specific implementation. 

The modification circuits 60, 61 add or subtract 
samples to the signals contained within the memories 56, 57, 
respectively. This modification of the samples can occur while 
the samples are being fed into the memories 56, 57, while they 
are being removed from the memories, or otherwise. 

As previously described, the rate of digital sampling 
is normally dependent on the complexity and frequency of the 
signals. In general, the more complex the signal and/or the more 
this conversion ratio is removed from unity, the more samples 
will be needed. The reason for this is that using too few 
samples in a complex, high frequency conversion will result in 
excessive distortion. For example with a 30MHz computer square 
wave high/low input signal, three samples per clock period would 
be sufficient to slow the signal down to 20MHz (see fig 16) . 
However, with a 20kHz audio signal, a sample rate much higher 
relative to signal frequency would be necessary to provide a 5 
percent expansion without audible artifacts (see representative 
fig 17) . This is because the information content of the computer 
signal is accurately conveyed solely by the on/off state. Thus 
the digital sampling rate must be carefully selected in view of 
the signals, both input 10 and output 100, as well as the 
difference ratio 51 to be encountered. 
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Changing over between the signal of the first memory 56 
and the signal of the second memory 57 is accomplished by a reset 
control 65. When the reset control 65 operates, the signal 
modification circuit 50 changes over to the previously inactive 
memory circuit to remit the output signal 100. The other once 
active memory at the same time becomes disconnected from the 
output. The result is to increase or decrease the effective 
length of the overall memory. 

Note, if samples are taken out of the memory more 
slowly than they are put in, the delay length will slowly 
increase to maximum. At maximum delay, there will be no 
available memory for the next sample to be put into. Conversely, 
if samples are taken out faster than put in, the delay will 
shorten to nothing. At zero delay, there will be no stored 
sample to be taken out. It thus is desired to reset the memories 
before they are full or empty respectively, and preferably so 
that whole cycles of signal are added or deleted. By resetting 
or changing over memories, a portion of the signal will be 
repeated, preferably an integral signal, or a portion of the 
signal will be lost, again preferably an integral cycle. The net 
gain in either case, however, is to vastly increase the amount of 
apparent memory. 

The reset control 65 operates by comparing two signals 
in order to determine their similarity, with the changing over 
between the signals in memory occurring based on this similarity. 
The purpose of the reset control 65 is to maintain the overall 
throughput delay within acceptable limits. The signals being 
compared typically should include a delayed signal and another 
signal, which may itself be also delayed or not. For example, in 
a compression circuit, samples are being removed and a signal is 
effectively output from memory faster than it is being input. 
For this reason, the reset point could be at maximum delay or at 
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the end of the memory. A delayed signal at this point is then 
compared to the relatively undelayed signal that is being used by 
the modification circuit for changeover to maximum delay. 
However, in an expansion circuit, samples .are being added and a 
signal is effectively output from memory more slowly than it is 
being input. For this reason, the reset point could be at 
.minimum delay, or at the beginning of the memory. A relatively 
undelayed signal at this point {which could even be the input 
signal) is then compared to a relatively delayed signal that is 
being used by the modification circuit for changeover to minimum 
delay. As an additional example, the reset control 65 could be a 
computer that has as a signal feed of only the input signal, 
which the computer compares with time shifted versions of such 
signal to operate this reset control 65 through analytical 
analysis, essentially comparing two versions of the single input 
signal for similarity; this with no other direct connection to 
any signal modification circuit signal input or output. As a 
further example, the reset control could be a computer that 
compares a plurality of signals over a range at one end of a 
memory with a second plurality of signals over a range at the 
other end of a memory, with the reset occurring between the two 
most similar signals. In this device, the signals being compared 
would be developed from an analysis of a plurality of signals. 
While the resulting reset might not produce the maximum amount of 
available memory {i.e., be to the end of the memory), complex 
signals would be effectively processed. Therefore, depending on 
circuit design, the signals compared for similarity may vary from 
that disclosed herein. The key is that the signals have a 
statistical probability of being similar and are representative 
of signals displaced from each other in the memory. For example, 
the reset control could compare: a) the signal at the output of 
one memory 56, 57 to the input of the other memory 56, 57; b) the 
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1 output 100 to the input 10; or, c) otherwise as desired or 

2 appropriate. In the preferred expansion embodiment of this 

3 specification, the input signal and output from the active memory 

4 are compared. In the preferred compression embodiment, the 

5 output from the active memory and the signal delayed relative to 

6 this output are compared. 

7 The reset control 65 in ascertaining similarity, 

8 preferably, compares the signals for: a) relative slope between 

9 signals; and, b) relative amplitude between signals. These are 

10 compared to preset thresholds. Signals meeting the criteria of 

11 low relative slope and amplitude include periods of no 

12 information (i.e., silence in audio). Video signals could be 

13 . compared for scene change points for the addition/deletion of 

14 frames in a video image in an alternative embodiment. Video 

15 signals could also be compared for static video images, in an 

16 alternative embodiment. 

17 The Algorithms used to extract signal comparison 

18 information must be selected consistent with the type of coding 

19 used. It is preferred that the compared signals be 

20 examined for similarity in as many characteristics as practical. 

21 For example, the use of zero crossover in a similar direction 

22 alone in a reset control could produce an artifact if one signal 

23 was a high frequency signal at that point while the other was a 

24 low frequency signal. Similarly, least relative slope alone 

25 could cause unacceptable artifacts to critical listeners of an 

26 orchestra if resetting occurs more rapidly than is absolutely 

27 necessary. Look forward and look back comparisons would thus 

28 preferably be utilized in the reset control 65 so as to optimize 

29 the overall comparison with time displaced information as well as 

30 current signal status. Further, the reset control 65 would 

31 preferably include an override in response to these overall 

32 comparisons in order to limit unneeded resets, thus optimizing 
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the comparison procedure. In general, the more comparison 
attributes are included, the better the operation of the reset 
control 65. 

The reset function is often accompanied by a repeat of 
signal (pitch increase mode; samples removed) or discard of 
signal (pitch decrease mode; samples added) . 

Using an example memory device that is capable of both 
modes (fig 31) with two parallel memory lines 56, 57 and two 
signal modification circuits 60, 61, the reason for this signal 
repeat /deletion can be readily understood. In this device, an 
input signal 10 is being continually fed into the beginning of 
two RAM memory lines 56, 57. 

In the pitch increase mode A, the active signal 
modification circuit (60 shown) uses signal data at a somewhat 
faster rate than it is being fed into the memory line 56 (the 
actual rate dependent on the expansion ratio) . This causes the 
active signal modification circuit 60 to relatively advance up 
the memory line 56. While this is occurring, the reset control 
65 is examining two signals for similarity. In the embodiment 
shown, the signal being output by the active signal modification 
circuit 60 and the signal at the circuits reset position are 
compared (as depicted by dotted lines in all figures) . When the 
signals are similar (or when the active memory line 56 is used 
up), a reset occurs. This causes the other memory line 57 to 
return to the relative reset point and other modification circuit 
61 to become active. However, since the input signal 10 has been 
continually fed into both memory lines 56, 57, this reset causes 
a certain portion 58 of the input signal 10 to be processed for a 
second time repeating this data at the output of the signal 
modification circuit 50. As the input signal 10 is periodic, and 
an integral number of signal periods is repeated, the artifact is 
acceptable. For example, as shown in the FIGURES 23-25, the 
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actual repeat would normally be one or more complete cycle of the 
input signal. The actual reset would repeat only a fraction of 
the actual signal content. In fact, this fraction is typically 
equal to the compression factor. 

In the pitch decrease ,mode B, the active signal 
modification circuit (60 shown) uses signal data at a slower rate 
than it is being fed into the memory line 56. This causes the 
active signal modification circuit 60 to relatively retreat down 
the memory line 56. While this is occurring, again the reset 
control 65 is examining two signals for similarity. In the 
embodiment shown; the signal being utilized by the active signal 
modification circuit 60 and the signal at the circuits reset or 
input position (dotted lines in all figures) are compared. When 
the signals are similar (or when the active memory line 56 is 
used up), a reset occurs. This causes the other memory line 57 
and other modification circuit 61 to become active. However, 
since the input signal 10 has been continually fed into both 
memory lines 56, 57, the reset causes a certain portion 59 of the 
input signal 10 to be never processed, thus deleting this data 
from the output of the signal modification circuit 50. Again, as 
the input signal is Periodic, and an integral number of periods 
is deleted, the artifact is acceptable, especially given the ways 
described herein of reducing its noticeability . 

Note this example uses two memory lines 56, 57 and two 
modification circuits 60, 61 with a single sample rate for 
clarity of explanation. In other embodiments, the memory lines 
56, 57 could be combined, a single modification circuit could be 
utilized, only increase or decrease, or both increase and 
decrease, could be provided in a single circuit, the processing 
could occur in real time for pitch shifting and other signals 
could be compared for reset control. Further, the signals in 
memory could be clocked out at higher or slower rates than being 
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input, thus providing the alteration of the alpha length of the 
input signal without movement of the signals to the modification 
circuits (which in both compress and contract could be located at 
the end of twp individually differing clocked memory lines) . 
Examples of these and other embodiments are given here and 
elsewhere in this application. 

The resetting increases or decreases the effective 
length of the overall memory with acceptable artifacts by 
changing over between effective memories as previously described. 

In the preferred embodiment, when the chosen 
parameter (s) are similar, the reset control 65 changes over 
between the relative memories 56, 57 as described. In addition, 
the previously inactive memory 56, 57 is reset at the time of 
changeover, thus extending its relative length. 

Note that if there are multiple related channels, such 
as stereo audio. or surround sound, this switchover preferably 
occurs when all important channels are similar at the same time, 
thus minimizing perceived stereo or spatial phase shift . This 
reduces the loss of stereo or spatial imaging. This correlation 
is preferably occasioned by logically connecting each channel 
associated reset control 66 to the reset control 65 for 
cooperative signals in order to provide for comparisons of their 
respective signals similarity. 

In addition, in a possible alternative embodiment, if 
the reset control 65 has not operated near to either end of an 
operative total memory length, changing over is forced;' 
preferably based on some sort of optimization formula (ideally 
computerized) . This preferable forcing recognizes that a 
particular memory length may run out of its ability to compensate 
for the delay between the compared signals prior to the compared 
signals being similar (as previously set forth) , and thus might 
produce annoying artifacts. Note also that if there are multiple 
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1 channels (for example again stereo audio) , the signals might be 

2 forced independently according to individual parameters. One 

3 example of this would be if one channel had high frequency (such 

4 as a fife) , and the other channel had low frequency (such as 

5 drums) . The optimum forcing times for each respective channel 

6 might not be coincidental. In this particular example, the low 

7 channel might be forced at a different time from the high 

8 channel: if the channels were forced simultaneously, there could 

9 be high frequency cancellation and/or loss of imaging. It is 

10 therefore appropriate for the forcing to be under the control of 

11 some logic specifically designed for the nature of the signals. 

12 For example with off /on computer signals, the two channels could 

13 be forced independently: Each according to its own parameters. 

14 However, for a stereo audio system or for an audio signal 

15 synchronized with a video image, care must be taken not to 

16 destroy the imaging and/or synchronization. The channels must 

17 therefore be correlated to insure that this does not happen. 

18 The circuitry in FIGURES 22 and 26-27 have a reset 

19 control 65 which compares the intermediate signals in the 

20 modification circuits 60, 61 for similarity with the reset 

21 control 65 operating a cross fade control 70 to change over 

22 between modification circuits 60, 61. 

23 Note if the length of the delay in the memories 56, 57 

24 was appropriately extended, the changing over could be 

25 artificially manipulated including to allow for the deletion or 

26 insertion of blocks of information while also reducing the 

27 artifacts to a minimal amount. As an example of this, in talk 7 

28 show profanity memories, it would be possible to change over to 

29 the inactive memory at the immediate end of the time of the 

30 profanity, thus in effect resetting the profanity memory without 

31 any additional loss of signal. As an additional example, a 

32 documentary could add to or delete audio independently of the 
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video content by increasing or decreasing audio delay without 
introducing an annoying pitch change. Further, if there is a 
correlation between two signals, the use of the invention would 
allow one to independently resync the signal's correlation 
without introducing objectionable artifacts. 

An example of how the input 10 and output 100 signals 
may correlate will now be described in example form. In the 
embodiment of the invention now specifically described, the input 
10 and output 100 will be analog signals with frequency shifting 
alone occurring. 

The input signal 10 is a signal stored for production 
over a first length of time. This input signal 10 "has frequency 
related information thereon. An example would be a television or 
audio signal from a video cassette recorder or other storage 
means. A simplified stylistic version of this input signal 10 is 
shown in FIGURE 9 {in the actual signal, a more complex waveform 
would normally occur; see FIGURE 12 for example) . For reasons 
not particularly important to this example, it is desired to 
extend this example signal by substantially 5.6 percent without 
altering the apparent frequency content thereon. 

Note that figures 9-21 are given by way of example. 
Other sampling/modification methods could also be utilized with 
the invention. Note also that for clarity of explanation in 
these figures that the input sample is converted to digital by a 
leading edge sample and hold circuit (left edge) , while the 
output sample is converted to analog by a trailing edge 
conversion circuit (right edge). Alternate conversion circuits 
could be utilized if desired. For ease of comprehension, no 
interpolation is used in figures 9-21. 

The first step of this example is to replicate the 
input signal 10 of FIGURE 9 into digital form. In the . devices of 
FIGURES 22 and 25 this is accomplished by a pulse amplitude 
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1 analog to digital converter 14 . An example of this digital 

2 sampled signal is shown in FIGURE 10. (Although pulse amplitude 

3 modulation is shown for this digital example, it is to be 

4 understood that other coding methods could be utilized without 

5 deviating from the claimed invention. Examples include PWM, PEM, 

6 PDM, PCM, PPM, PNM, PFM, PLM, and PIM. 

7 In this FIGURE 10, the analog signal 10 has been 

8 replicated into digital form 20 through the use of PAM, 

9 specifically thirty six (36) digital samples 21, each having an 

10 amplitude. In that the input signal 10 is to be compressed by 

11 5.6 percent, this 5.6 percent is the equivalent of two samples of 

12 our example digital wave 20 of FIGURE 5 (it is necessary then to 

13 add two digital samples 22 to the input signal 10 in order to 

14 frequency compensate such signal) . This exemplifies the fact 

15 that in the preferred embodiment the sampling rate preferably 

16 must be high enough so as to allow the insertion (or deletion) of 

17 a sample 21 in the replication of the output signal 100 , without 

18 introducing unacceptable distortion. 

19 In general, the higher the sampling rate, the less 

20 distortion will be introduced by the frequency conversion 

21 process. This is particularly true in respect to the frequencies 

22 where the consumer is most sensitive. In addition, it is 

23 preferred that the samples 20 be inserted (or deleted) where any 

24 artifacts would be least noticeable. In the case of a 

25 significant expansion (or contraction) , the samples would 

26 preferably be spread out over the entire length of the wave form 

27 (see fig 12 for a representative complex wave form) . 

28 Our example signal is a sine wave with the samples 

29 added at the point of least slope; the peaks 23 of the positive 

30 wave and negative wave, respectively. Therefore, needing only 

31 two samples, at these two points the signal modification circuit 

32 50 inserts an additional sample 24 in both the positive and 
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negative series of digital samples 21. This produces the 
modified digital signal 25 of FIGURE 12. A modified signal 
without highlighting is shown in FIGURE 13 ; now representative of 
the sampling 120 of the output signal 100. Note that FIGURE 7 
has a peak sample repeated. 

When the compressed digital representation of the 
signal 120 in FIGURE 12 is taken through a digital to analog 
converter 114, the result is the signal shown in FIGURE 14, an 
output signal 100 having an effective alpha length 113 some 5.6 
percent longer than the input signal 10 shown in FIGURE 4 . The 
difference between the two signals 10, 100 is shown in FIGURE 15. 

Due to this difference, the output signal 100 can be 
played back at a clock rate some 5.6 percent higher than the 
input signal 10 in FIGURE 9 while at the same time producing a 
signal having the same frequency content to the observer as 
existed in the input signal 10. (Compare fig 9, i.e., the input 
signal 10, with fig 16 wherein the output signal 100 is clocked 
at a rate 5.6 percent higher than fig 9: The two signals produce 
the same frequency content . ) This would allow a television 
station to shorten the time of a television program accordingly 
without increasing the pitch of the related audio information. 

It should be noted that in this example the 
compression/expansion factor is equal to the inverse of the 
number of samples between peaks. In the more general case, where 
such a simple relationship did not exist between waveform period 
and pitch shift factor, it typically would not necessarily be 
possible to arbitrarily select the more favorable points for 
sample insertion or deletion. 

It should also be noted that in cases of types of 
coding differing from the PAM of this example, differing sample 
modification algorithms may be required. 

The difference between our example input 10 and output 
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signal 100 is some 5.6 percent. However, any difference, 
including this 5.6 percent, is cumulative for each cycle of the 
input signal 10. For this reason, the memory necessary to 
perform the function of modifying the input signal 10 into the 
output signal 100 can be computed by multiplying the number of 
repetitive cycles by the difference time factor, by 5 . 6 percent 
in our example. This adds up very quickly to a significant 
amount, an amount requiring ever increasing memory. 

To avoid this, the applicant's invention uses the reset 
control 65 to change over between effective memory points in the 
signal modification circuit 50 so as to reuse the same effective 
memory repeated times. 

This reset control circuit 65 operates when the two 
signals being compared are similar within prescribed parameters 
as previously set forth. It then changes over between similar 
signals. An example of the comparison can be seen in FIGURE 17. 
This figure represents a circuit adding samples to a first analog 
signal 63 to produce a second, expanded analog signal 64. In 
this figure, after 38 cycles, the first analog signal 63 and the 
second analog signal 64 both have a positive going similar slope, 
same direction, zero crossings at the same location 70. If one 
immediately reset the signals, and began the cycle anew at this 
time, an individual to whom the signal 100 was addressed would 
not notice any significant artifacts. For this reason, the 
switchover could compensate for the increasing phase lag between 
the signals with memory having a finite length. The cost of this 
in FIGURE 23 is the loss of one cycle of the first analog signal 
63, which one cycle would never be produced to be perceived by 
the individual. 

As previously set forth this resetting is most easily 
accomplished by changing over between two memory lines 56, 57 
accompanied by a resetting of the second memory line (as was 
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further described in respect to FIGURE 22) . While this 
compromises the input signal 10 information, this resetting 
significantly reduces the amount of memory necessary for the 
device to operate. More importantly, even if infinite memory 
were available, resetting must still be done. Otherwise, the 
desired effect of time compression or expansion would be 
completely canceled. As an example of this, memory line 56 would 
be utilized for the time 67 with memory line 57 utilized for the 
time 68 to produce the output 64. 

FIGURE 24 represents a circuit deleting samples of a 
first analog signal 63 to produce a second compressed analog 
signal 65. This figure is a graph comparing the first analog , 
signal 63 and the second analog signal 65 and illustrating the 
reset 140 of the signals at a common amplitude, similar frequency 
signal peak. At this time, again one would change over between 
the two memory lines 56, 57 and simultaneously reset the second 
memory line. This has the effect of adding to the output 100 by 
repeating one cycle of the input signal 10 information, again a 
resetting significantly reducing the amount of memory necessary 
to operate unnoticeably as well as maintaining through delay 
within acceptable limits. Again the two memory lines 56, 57 
would preferably be utilized alternately to produce the output 64 
without seams. 

The invention finds particular application in MPEG 
audio and video compression (figs 28-29) . In the MPEG system, 
there is a considerable amount of video processing which goes on 
which is dependent upon the complexity of the video image which 
is compressed and/or decompressed. This is also true for the 
various audio signals which accompany the video. Due to the 
differences in compression complexity, there is frequently a 
corresponding variable delay interval in compression and/or 
decompression time which causes missynchronization of the audio 
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1 and video signals when they are ultimately decompressed for use, 

2 as for example by receipt of an MPEG compressed HDTV television 

3 signal by a viewer. 

4 In order to attempt to overcome the . audio to video 

5 asynchrony problem, the MPEG compression standard provides for 

6 encoding a unique number in the compressed audio and video data 

7 stream every 0.7 seconds or so at the time of compression. Upon 

8 subsequent decompression, these numbers are presented to the 

9 audio and video system decoders in order to facilitate a 

10 comparison of the audio number to the video number in order to 

11 allow the two to be brought back into relative synchrony at' these 

12 periodic times by manipulating the video via video frame 

13 memories. This still, however, can produce ascertainable 

14 artifacts tiring or objectionable to the observer. Further, 

15 multiple frame video memory buffers are necessary to allow the 

16 needed frame comparisons to repeat or drop identical frames. 

17 This memory is expensive and the processing complex. 

18 The above missynchronization of the audio and video 

19 signals of a particular entertainment program can be a serious 

20 problem, especially when the audio leads the video, as this is an 

21 unnatural condition and leads to conscious or unconscious stress 

22 in the viewer. This is an unfortunate byproduct of the MPEG 

23 compression. The unnatural audio leading and/or trailing video 

24 condition is also known to diminish the viewer ! s perception of 

25 the quality, for example the entertainment value of the program 

26 being viewed. 

27 The present invention can be utilized to resynchronize 

28 the signals, thus eliminating the tiring and objectionable 

29 artifacts. Further, the multiple frame video delay and complex 

30 comparisons of the MPEG standards are not needed even though 

31 better synchronization is achieved. The invention accomplishes 

32 this by effectively, speeding up or slowing down the audio 
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signal (s) to synchronize it to the video, and does so without any 
pitch change. 

The invention preferably utilizes the automatic 
reference signals to accomplish this synchronization. The two 
preferred reference signals which are encoded at the time of MPEG 
compression are SCR (System Clock Reference) and PTS, 
(Presentation Time Stamps) , the details of which may be found in 
the MPEG specification standards published by ISO/IEC. Other 
compression standards suffer from the same problems and may make 
use of these or similar SCR or PTS type schemes, which will be 
referred to collectively here as time flags. 

The encoded time flags thus represent the starting time 
at which a particular video frame and its associated audio 
signal (s) arrive at the encoder, and/or are to be played back 
together from the decoder. 

In this example MPEG system as it presently exists, if 
the video decoder sees that the video frame it is about to play 
back has a code which is later than that is, occurred after, the 
audio which is currently being played back, a frame of the video 
is repeated to bring the two close to synchrony. If the video 
frame is before the audio which is currently being played back, a 
frame is discarded. Such action, which is suggested by the MPEG 
standard, finds only limited capability in preventing audio to 
video asynchrony. Further, there are other problems with the 
frame drop/frame repeat method of achieving synchronization. 
First, adjustments may be made only in one frame increments, 
giving rise to potential residual errors. Also, the required 
video memory is costly compared to an equivalent amount of audio 
memory. Further, it may be that it is needed to discard a frame 
of video to synchronize signals, but the full frame might not yet 
have arrived due to complex decoding requirements. More than one 
frame of adjustment might also be needed, giving rise to the need 
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for even more video memory with the associated cost problems . 
Since adjustment is made in frame jumps, this can also cause 
motion related artifacts, and, if the relative delay is constant- 
ly changing, the system can alternately repeat and delete frames 
of video causing an artifact known as motion judder (jitter + 
shudder) . Further, the missynchronization that does exist could 
cause subconscious or even conscious ascertainable artifacts, 
reducing the viewer's enjoyment. 

The invention of this application can be utilized in 
conjunction with a continuously variable audio delay to provide a 
better audio to video synchronization than in the present MPEG 
frame drop/frame repeat standard. Further, this is achieved at a 
lower cost. The invention preferably accomplishes this by adding 
samples to the audio to allow the audio to be slowed down to re- 
synchronize the signals (in the instance of advanced audio) , or 
deleting samples to allow the audio to be sped up to re- 
synchronize the signals (in the instance of delayed audio) . The 
signal modification circuit may be used by itself or in 
conjunction with a frame drop/frame repeat device. 

The video signal to be output from the receiver's video 
decoder precedes the audio. This delay allows the processing of 
the video signal to synchronize the eventual audio signal output 
with the video signal. (The resulting correspondingly advanced 
audio is delayed in the receiver to allow synchronization to be 
achieved.) As previously discussed, this synchronization is 
accomplished in the MPEG standard by dropping or repeating video 
frames . 

In the device of FIGURE 28 incorporating the present 
invention, the MPEG input signals are fed from the normal 
compressor encoder circuits 160 in the customary mode of 
transmission (tape, transmitter, receiver,' caLble, etc.) to a 
modified adaptive decoder 190. The adaptive decoder 190 shown is 
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modified from a customary MPEG adaptive decoder in that the 
adaptive decoder 190 has only slightly over a single field of 
video memory 191, and the use of this optional memory is to allow 
for an unusual override reset of the signal modification circuit 
150 (as later described) . (In contrast, the customary MPEG 
demodulator has at least two frames of memory, which memory is 
actively used for audio resynchronization . ) 

The video signal in this modified device is directed 
through the adaptive decoder 190 to produce an output video 
signal in the customary manner according to with MPEG standards, 
except that it trails the audio signal by approximately two 
frames (this due to the deletion of the audio to video 
synchronization from the video signal path) . 

The audio input signals 10 are fed through the adaptive 
decoder 190 to convert such signals to customary form. The audio 
input signals 10 are, however, then passed through a signal 
modification circuit 150 in order to add or subtract samples to 
synchronize the audio to the video signals. This synchronization 
is accomplished by adding or deleting samples in the audio input 
signal 10 according to the invention. Thus, as the relative 
delays of audio and video change, the audio delay time may be 
adjusted, without pitch changes, to enable proper synchronization 
with the video. This is preferred in that it occurs at a much 
lower hardware/ software cost than a video memory or memory system 
would produce. This would normally be an intermittent procedure 
accomplished after relative synchronization was lost, typically 
during times, of major video changes. The relative video delay 
allows resynchronization without the introduction of 
objectionable artifacts. For example, the unique number 
transmitted every .7 seconds in the MPEG system could provide a 
reference for automatic synchronization by altering the signal 
modification circuit 150 to add or subtract samples as needed 
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under the control of the interface comparing logic 170 i 

The control of the audio delay may be either a feed 
forward (fig 28) or a feed back arrangement (fig 29) . 

In the feed forward configuration, the time flags of 
the video and audio which are output from their respective 
decoders are compared to determine the amount of delay which the 
audio needs to achieve synchronization with the video. This 
delay is coupled to the audio delay control to cause it to change 
to the desired amount. In this application, the audio and video 
signals are fed through a compression encoder 250 subject to the 
time flag 251 in the customary manner. However, on decompression 
while the video signal 10 is fed through a customary rate 
convertor 253, the audio signal 10 is fed through the signal 
modification circuit 50 to resynchronize the signals. A 
comparator 255 analyzes the video flag and audio flag to 
automatically control the sample addition/deletion of the 
modification circuit 50. 

In the feed back configuration, the time flags of the 
audio output from its decoder are delayed by the same amount as 
the audio is delayed in the variable memory. The audio time flag 
and audio may of course be delayed in the same, or separate 
matching memories. The video time flag corresponding to the 
output from the variable memory is compared to determine the 
amount of synchronization error of the audio relative to the 
video. This error is coupled to the audio delay control to cause 
it to change to correct the error. In this application, the 
audio signal 10 is fed to a modified signal modification circuit 
50A, a circuit that actively acts on the audio flag in addition 
to the audio signal 10. The comparator 255A then compares this 
delayed audio flag to the video flag in order to control the 
signal modification circuit 50A to resynchronize the audio output 
to the video. 
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In this resynchronization, however, as previously 
described in respect to FIGURES 17-19, there is a possibility 
that very occasionally the reset control 65 in the signal 
modification circuit 50 might not have operated near to the end 
of an operative total delay length. In FIGURE 23, when this 
would occur, the field memory would preferably be operated in 
order to repeat a field, and thus allow for the forced resetting 
of the signal modification circuit 50. In the embodiment shown, 
this is provided by an override signal 151 from the signal 
modification circuit 150 to the video decoder. 

As a further improvement, it will be advantageous to 
either compare time flags of multiple sound channels of a given 
audio signal in order to correct any channel to channel phasing 
and polarity errors which may exist (fig 30) . One example of 
such a system would be video surround sound audio having 5 
channels corresponding to left front, right front, left rear, 
right rear, and subwoofer. The channels would be compared to 
each other to ensure that their relative phasing or timing are 
kept correct, with any timing errors being used to control fine 
phase adjustment of each controlled audio channel. In this 
application, the various channels 1-5 would each be subject to 
its own individual delay in the delay circuit 56B, with samples 
added or subtracted therefrom based on a signal modification 
circuit SOB. This circuit SOB and the delays are automatically 
adjusted by the flag comparison circuit 255B. Feed forward could 
also be utilized. 

The control of multiple sound channel timing for any 
multiple sound channel application may be implemented by itself 
with the variable delay capability of the present invention. 
This is especially true because of the precise delay control 
which may be achieved in the present invention. Multiple sound 
channel timing control is, however, quite cost effective. 
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Further, it is very useful to include such correction as an 
additional capability of audio to video synchronization 
circuitry. Another invention showing correlation of multiple 
audio channels for applications where audio and video are 
transmitted over different paths is shown in Cooper U.S. Patent 
4,703,355 which is incorporated herein by reference with respect 
to its prior art teachings and in particular, signal correlation 
and generation of control signals responsive thereto. 

The invention may be utilized in an entertainment or 
other system to provide faster or slower than normal recording or 
replay of audio, and if desired associated video. Other 
information may be utilized as well, with the teachings herein 
being just as applicable to storage or replay of any information 
having a frequency parameter where it is desired to alter the 
time duration without altering the frequency. 

FIGURE 31 shows a system in which a physical . storage 
medium 300 along with its associated scanning mechanism is 
controlled by the user by providing a varying reference to the 
scanning mechanism. It is preferred that the physical storage 
medium be a digital video disk such as a common optical CD device 
which utilizes a spindle motor to rotate the storage medium, 
which is the disk, and a laser and optical scanning head which 
make up the scanning mechanism. ■ It is preferred to provide the 
servo mechanism for the spindle motor with a variable frequency 
reference signal which is provided by a Numerically Controlled 
Oscillator (N.C.O.) 305 which frequency is controlled by the user 
interface and control logic 310. 

The user interface and control logic 310 also 
interactively controls the optical scanning head position, 
receiving positional data from the scanning mechanism (which 
alternatively may be provided via the recorded data) to perform 
start, stop, record, play, search and other functions normally 
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provided. In this manner the recording or playback of the audio 
and/or video data may be controlled to take place at different 
and variable rates. It should be noted that while the term data 
is used to denote what is stored in the storage medium, that it 
is not to be construed that this data is to be limited to digital 
data. Data as used herein is simply meant to mean the 
information which is stored in whatever form it may exist. 

While this embodiment of the invention is preferred to 
be a CD device, one skilled in the art will recognize from the 
teachings herein that any storage device may be adapted to 
implement the invention, including analog video disk recorders, 
analog audio tape recorders, DAT recorders, video optical disc 
recorders, video tape recorders, rotating magnetic disk recorders 
(such as computer disk drives) , film based recorders and 
projectors, solid state memory including semiconductor memories, 
charged coupled device memories, switched capacitor recorders, 
and three dimensional solid memories such as laser addressed 
crystal lattices. All of these storage device types have in 
common a mechanical, electronic, optical or combination scanning 
mechanism which selects where within the physical storage medium 
the data is stored or read. It is by controlling the rate at 
which this scanning mechanism operates that the rate of storage 
or reading is made variable. The invention may be utilized with 
record only, playback only or record and playback versions. 

As one skilled in the art will understand from these 
teachings, varying the rate of reading or writing may not be 
successful over a very large range by simply causing the scanning 
mechanism to operate at different rates. In virtually all 
storage devices, the data is stored in a form which takes on many 
analog characteristics. With the preferred optical disk medium, 
the data is stored as alternating light polarization of a layer 
of material. The data is read back by the laser beam which beam 
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is directed to an electro-optical sensor or converter thus 
providing an electrical signal which varies in response to the 
laser light. By changing the speed of the disk and scanning, the 
frequency and intensity characteristics of the laser light are 
altered, thereby altering the frequency and intensity 
characteristics of the corresponding electronic signal . 

In virtually all recording and playback systems, there 
is required compensation circuitry to make up for nonlinear 
characteristics of the system. These compensation systems are 
critical to the proper recovery of the audio and video and are 
included in the modulator 3 2 OA and demodulator 320 circuitry. 
Unfortunately, the nature of these nonlinear characteristics is 
dependent on the scanning speed, thus causing a serious 
compensation problem which is overcome in the above system. Most 
commonly, frequency response of the system changes in fashion 
which is dependent upon the frequency of the data. Many systems 
operate to detect the changes in the data rather than the data 
itself. The sensors which are thus used most commonly respond to 
the electrical, magnetic, electromagnetic, or optical flux from 
the storage and scanning mechanism combination and have an output 
which is proportional both to the magnitude and rate of change of 
the flux. The demodulator 320 thus must condensate for the 
change which compensation in turn must be altered as the rate of 
change of flux is altered by the changing of the scanning speed. 

The requirement to change the compensation caused the 
demodulator 320 to be adaptive, and controlled by the user 
interface and control logic 310 in order to properly demodulate 
the audio and video at any of the continuously selectable 
playback speeds. FIGURE 33 shows an adaptive filter network 
which is operable to change the filter compensation to fit the 
playback or record speed. The filter utilizes varactor diodes 
350 to provide a voltage variable capacitance, thus changing the 
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filter frequency characteristics. The voltage which is applied 
to the varactors is provided by a digital analog converter 360, 
which in turn is loaded with the appropriate digital number for 
the particular speed being used, which loading is accomplished by 
the user interface and control logic 310. Linearization or 
voltage mapping circuits 351 are provided at the input and output 
of the filter and at the varactors 350 to compensate for any non- 
linearities therein. In operation the filter operates to reduce 
the high frequency content of the data signal to a proper level, 
the amount of reduction depending on the speed of the data. 

Returning to figure 31, as for any particular physical 
storage medium which would be preferred, the proper design of 
compensation at given fixed speeds is well known to those skilled 
in the art. It is believed that from the teachings given herein, 
those skilled in the art will also be capable of designing 
compensation which is variable over a suitable range of 
operation. In this manner the invention of the figure may be 
practiced such that the operator may select any speed within a 
range of speeds with the recording and playback of data of the 
storage medium operating properly. 

It should be noted that some existing audio tape 
recording systems have the capability to change frequency 
equalization of the audio signal as the tape speed is changed 
from one discrete speed to another, for example from 7.5 IPS to 
15 IPS. In addition, many of these tape recorders may playback 
at continuously variable speeds to facilitate searching. 
Applicant is unaware of any of these systems having continuously 
variable equalizers to insure properly equalized audio at all of 
the continuously selected speeds which the operator is capable of 
selecting. ' 

After the audio and video is demodulated by the 
adaptive demodulator 320, it is required to pitch correct the 
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audio under control of the user interface and control logic 310 
as by the circuitry described in the preceding figures, and to 
convert the rate of the video to standard rates in order that it 
may be viewed on standard viewing devices. The video rate 
converters 330 are known in the art as frame synchronizers. 
While it is possible for frame synchronizers to operate self 
contained, improved operation and lower cost may be obtained by 
providing the rate converter with information and control from 
the user interface and control logic, and additional features 
such as video special effects may be provided as well. The pitch 
converter 340 is as set forth in this present application. 

If it is desired to record at variable rates, the same 
problems as described above for playback enter into the proper 
modulation of audio and video before recording. Consequently it 
is needed to provide an adaptive modulator 320A. In addition, it 
may be desirable to provide a pitch corrector 340A and video rate 
converter 3 3 OA on the record side of the device in order that the 
recorded data appears as if it had been recorded at a particular 
rate. One skilled in the art will recognize that a desired end 
result of expanding or compressing a time segment may be achieved 
by operating on the video and/or audio either when recording or 
upon playback or both. In addition, there is no requirement that 
both audio and video be operated on at the same time, as one may 
be corrected on recording and the other upon playback. 

A two way pitch shifter has been implemented with a 
Star Semiconductor SPROC IC and is shown in FIGURE 20. The SPROC 
IC is available from Logic Devices, Inc. of Sunnyvale, California 
and is of type SPROC- 1400 -50PG132C. Information on the use of 
this and similar ICS is available from Logic Devices, Inc. 

The preferred embodiment would use type SPROC-1400- 
50MQ144C, which functions similarly, but at a lower cost. 

The block diagram FIGURE 26 is directly compatible with 
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the SPROC Development System Part Number SPPROClab tm version 
1.25, P/N SDS-1001-03 with SPROClab version 1.25. patch B, P/N 
SDS-1002-01B. Figure 26 is a schematic representation of digital 
signal processing. 

In this embodiment of the invention, the frequency 
converter system is specifically designed for a stereo audio 
source having a 20kHz input bandwidth. However, it should be 
understood that the frequency converter system is equally 
adaptable to systems of various input frequencies and bandwidths. 
The audio input signals 10 are sampled at 48kHz sample rate. The 
50MHz P/N SPROC-1400-50PG132C SPROC chip is used for processing. 
Left and right channels are processed identically, and in 
parallel. Some control blocks (or "cells" in the Logic Devices, 
Inc. nomenclature) are common to both left and right. 

Three basic operations are performed: stretch (or 
expand) /interpolate or compress/ interpolate , reset, and reset 
timing control. 

ST-INO 202 accepts Format 2 stereo input samples which 
are provided by a Crystal Semiconductor CS5326 A-D Convertor. 
Output 1 is the right channel, and output 2 is the left. This 
discussion will concentrate on the left channel and common 
processing. 

Stretch cells ST1 200 and ST2 201 each perform the 
stretch/interpolate or compress/interpolate functions in stereo. 
Most of the time, only one is actually connected to the output. 
Two are required during a reset operation, as will be described 
below. ST1 200 and ST2 20 1 perform reads and writes to RAM 
circular buffers 56, 57 in order to accomplish the stretch or 
compress function. 

. Two way stretch cells are used at STI 200 and ST2 201. 
In the pitch increase mode, delay might start at 35 mS and 
decrease until a crossfade reset event occurs. At reset, delay 
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is returned to or near 35 mS, and the process repeats. In the 
pitch decrease mode, delay starts at or near zero, and increases 
until a cross fade reset event occurs. At reset, delay returns 
to near zero, and the process repeats. Thus, while in the pitch 
decreasing mode some program material is discarded at a reset, in 
the pitch increasing mode some program material is repeated. In 
the pitch increase mode, ST1 200 and ST2 201 output stereo 
samples at the 35 mS memory candidate reset point for splice 
match testing. These samples, "OUTC" 203 and "OUTD" 204 of ST1 

200 (203A, 204A) and ST2 201 (203B, 204B) cells, pass through 
switches SW3 205 and SW4 206 so that those of the correct stretch 
cell may be selected. If the pitch decrease mode is selected, 
"OUTC" and "OUTD" produce samples of near zero delay. 

An abrupt reset of a single stretch cell often results 
in an audible click. In this design, a more gradual reset occurs 
as follows. Suppose only ST1 200 is connected to the output. 
Further suppose the pitch decrease mode is operative. At the 
start of a reset, ST2's 201 delay is set to or near zero. At the 
same time, a gradual fade from ST1 200 to ST2 201 is initiated 
and performed by 207. At the completion of the fade, only ST2 

201 is connected to the output. The delay through each stretch 
cell continues to change during the fade operation. At the next 
reset, the operation is reversed, and so on. This technique is 
effective at suppressing reset clicks. 

The left channel outputs of ST1 200 and ST2 201 connect 
to the output via MFADE0 207. A ramp signal is applied to MFADE 
207 to accomplish the fade. This ramp is formed by MMV1 211, 
MMV2 212, MINUS 3 217, and INT2 214. Depending on direction, 
either oneshot MMV1 211 or MMV2 212 produces a pulse 3072 samples 
(64mS) long. Integrator INT2 214 forms the ramp. Subtraction 
cell 217 causes alternate ramps to be in different directions. 
INT2 214 output takes values between 0 and 1.0. Ramp length of 
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64mS was determined empirically. Too short a ramp produces an 
abrupt transition; too long produces an echo effect. 

A reset is initiated only when the differences between 
input and output slope and instantaneous amplitude are within 
certain limits, in order to make the cleanest splice practicable. 
Amplitude and slope match for the left channel are measured by 
MINUS2 221; first difference cell DIFF2 222; and cells pi 223 and 
p3 224. SW1 225 selects the appropriate stretch cell output. pi 
223 or p3 224 produces an active low reset signal whenever the 
magnitude at the cell input is below a certain threshold. These 
thresholds are continuously scaled according to signal level and 
slope. The intent is to require the same relative match, 
independent of amplitude or instantaneous frequency. This 
scaling is performed by SUM1 213; DIFF3 230; and, peak detectors 
PDO 231 and PD1 232. SUM1 213 performs full wave rectification, 
of each input, and sums the results to produce its output. PDO 
231 and PD1 232 gains set the degree of match required. 

Outputs from pi 223 and p3 224, as well as outputs from 
the corresponding right channel cells p9 235 and plO 236, must 
all be simultaneously low to enable a reset. Or gate OR1 229 
performs this function. Additionally, cell p7 237 sets the 
minimum time between reset events at 9600 sample periods (200mS) . 
Any reset inputs to p7 within the last 200 mS from a reset output 
from p7 are ignored. The 200 mS value was empirically optimized 
as a compromise between rapid warble like artifacts and loss or 
repetition of program material . Minimum time between resets 
could be caused to vary randomly about the 200mS value, to reduce 
the periodicity of resets. Some artifacts are still detectable. 
Cells p8 247 and OR4 248 have been carried over from previous 
design iterations, but are not functional in this design. 

P7 237 emits a one sample width active low reset pulse. 
This is steered to the appropriate oneshot and stretch cells by 

G:\7434CIP -62- 06/08/9 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 



logic formed by threshold detector GT1 238; INV1 240; Or2 241; 
and, OR3 242. Delay cells DELAY2 245 and DELAY4 246 are intended 
to correct timing misalignments. 

Cells that are specific to right channel operation are 
DELAY3 249, SW2 250, MINUS4 251, SUM4 252, p9 235, plO'236, DIFF4 
252, DIFF5 253, PD2 254, PD3 255, RECT5, and pll 256. pll is not 
functional in this design. Stereo output is accomplished by 
parallel port output cells OUT0 257 and OUT1 258. Output 
parallel data is subsequently converted to serial format by 
hardware interfaced to the SPROC tm parallel port . This serial 
data is then converted to 8X oversampling format by a Nippon 
Precision Circuits Ltd. SMS 8 13 APT IC. The over sampled data is 
passed to two Analog Devices, Inc. AD1862N digital-to-analog 
converter ICS, to produce analog stereo audio output. Control of 
pitch correction factor and reset point is accomplished by 
changing the output levels of VR98 259 and VR99 260. To decrease 
pitch by 9.09 percent, VR98 level should be 2 (reset point of 2 
samples delay), and VR99 level should be 1.0. For no pitch 
change, VR98 level should be 2, and VR99 level should be 0 . 0 . To 
increase pitch 10 percent, VR98 level typically would be 1680 
(reset point of 1680 samples, or 35 mS, delay) , and VR99 level 
should be -1.1. AMP3 261 scales the pitch correction factor 
output from VR99 260 and passes it to stretch cells ST1 200 and 
ST2 201. The output levels of VR98 259 and VR99 260 may be 
modified by the SPROC development system, both during the initial 
building of the design, and during operation. Ideally, pitch 
correction factor and reset point may be controlled by an 
embedded microprocessor communicating with one of the SPROC 
ports. Serial output cell OUT3 262 sends the fade ramp signal to 
a PCM56 DAC on the development board. This signal is a useful 
telltale of system performance. This port is not supported in 
the production board hardware, but this signal could be sent to 

G:\7434CIP -63- 06/08/95 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 



the probe port ■ (which will be supported) by modifying the SPROC tm 
initialization code. 

Left and right channels are reset simultaneously, and 
must meet the reset criteria simultaneously. The lowest gain 
possible for PDO 231, PD1 232, PD2 254, and PD3 255 should be 
used. Experiments with program material showed that gains of 
less than 0.1 (for example, 0.08) were too low. . With gains of 
0.08, excessive time between resets were regularly observed. 
Program material tests at 9.09 percent pitch decrease mode, with 
gains of 0.1, indicate that most resets occur within 250mS, and 
that intervals beyond 300mS are rare (the corresponding memory in 
the stretch cell would be one tenth these numbers; at 9.09 
percent pitch decrease) . These gains, as well as the other 
parameters such as minimum reset interval and ramp slope are 
easily changed in software. 

The memories of ST1 200 and ST2 201 in FIGURE 20 are 
arranged as circular buffers. Data is continuously written to 
the inputs such that each buffer contains the most recent samples 
for a period equal to the memory length. In fact, the contents of 
each memory are the same, since each has the same input. In 
addition, OUTC 203A and OUTC 203B output the same data. The same 
is true for OUTD 204A and OUTD 204B. Therefore, an improved 
embodiment would be obtained by combining ST1 200 and ST2 201, 
and using a single memory. SW3 205 and SW4 206 would no longer 
be required. It may also be possible to combine the functions of 
OUTA with OUTC and OUTB with OUTD, respectively. A minimum of 
two separate memory outputs would still be needed, of course, to 
support the signal comparison and reset operations . Further code 
savings could be obtained by eliminating the NDXIN and NDXOUT 
outputs from ST1 200 and ST2 201, or a combined cell, since these 
outputs are not used in this embodiment. 

FIGURE 27 shows pitch correction integrated with the 
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delay function. The pitch corrector functions of FIGURE 25 are 
essentially cascaded intact with the delay function, except with 
the added ability to force or inhibit stretch cell 200 and 201 
reset. Cell TAU0 263 operates directly on the input samples, and 
produces delay. TAU0 263 changes delay in a manner identical to 
stretch cell operation, but without resetting. TAU0 263 utilizes 
external memory. Since pitch shift occurs during delay change, 
TAU0 263 output is passed to ST1 200 and ST2 201 for pitch 
correction. TAU0 263 also performs parallel port access for 
stereo sample output and microcontroller interface. Stereo 
output samples from MFADE0 207 are passed to TAU0 263 for output 
via the SPROC tm parallel port. 

Cell CNO 264 coordinates operation of the delay and 
pitch correction cells. TAU0 263 reports its current delay to 
CNO 264. CNO 264 reads stretch cell index values. The indices 
from the active stretch cell are selected by SW5 265 and SW6 266. 
CNO 264 computes the current stretch cell delay, and adds the 
TAU0 263 delay to determine total delay. This total delay is 
passed to TAU0 263 for parallel port output to the micro- 
controller. 

Target delay is read from the microcontroller via TAU0 
263. If total delay equals target delay, CNO 264 holds TAU0 263, 
ST1 200, and ST2 201, by writing zero to the slew factor inputs 
of those three cells, and inhibiting stretch cell reset. 

If total delay differs from target delay by more than 
20 mS, the following sequence of events occurs. First, the 
stretch cells are initialized. If the delay change direction 
should be opposite of the previous change,, it is likely that the 
initial stretch cell delay is too far removed from the 
appropriate reset point. CNO 264 sweeps the reset point values 
in the vicinity of the nominal for a period of 0.5 Sec, enabling 
a high probability of a splice. When this splice occurs, the 
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stretch cell becomes initialized without an abrupt transition. 
Then, CNO 264 outputs complementary slew values to TAUO 263, STl 
200, and ST2 201. This produces delay change and pitch 
correction at a ten percent rate. STl 200 and ST2 201 delay the 
slew factor by the same amount as the audio, to maintain temporal 
registration between the two. The total delay change occurs 
primarily during fades, because of the pitch corrector reset 
action. Delay change during each fade is generally of slightly 
more than 20 mS in size. When the total delay becomes within +/- 
20 mS of the target, the stretch cells are held, and TAUO 263 
slews at a 0.2 percent rate, without pitch correction, until the 
target is reached. If TAUO ' s 263 delay reaches zero before the 
target is reached, TAUO 263 is held, and the stretch cell delay 
is decreased at a 0.2 percent rate, without pitch correction, 
until the total delay is correct. 

For delay changes of 20 mS or less, the delay slews at 
a 0.2 percent rate, without stretch cell initialization or pitch 
correction. 

The minimum delay target is one sample (1/48000 Sec), 
not counting data converter decimation or interpolation filter 
delay. Dump mode is enabled by requesting zero delay. In dump 
mode, TAUO 263, STl 200, and ST2 201 are reset to minimum delay 
and held. Requesting a nonzero delay target causes normal 
operation to resume. In the present embodiment, control via the 
SPR0C tm development system is substituted for microprocessor 
control . 

In FIGURE 27 STl 200 and ST2 201 could also be combined 
in a manner similar to that described for STl 200 and ST2 201 of 
FIGURE 26. As before, SW3 205 and SW4 206 would not be needed. 
In a combined stretch cell, only a single output NDXIN would be 
required, which would also eliminate. SW5 265. 

Further reduction in code would be obtained by 
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combining ST1 200 and ST2 201 not only with each other but also 
with memory cell TAU0 263. A single memory would then be used. 
In both FIGURE 26 and FIGURE 27, a smoother fade ramp could be 
obtained by using a raised cosine function, instead of a linear 
ramp. In an improved embodiment, a watchdog timer function would 
be incorporated into the hardware and code . 

The preferred embodiment of the present invention may 
be modified without deviating from the invention as claimed. 

For example, the preferred embodiment is described as 
operating on the input signal, a signal which is being produced 
at a first speed, to produce the output signal, which output is 
subsequently clocked at the second speed to provide the same 
frequency based information. It would be possible to modify the 
order of conversion without deviating from the invention as 
claimed. For example, one could modify the frequency conversion 
system of the present invention to create a second plurality of 
signals clocked at different speed prior to frequency conversion. 
This could occur, for example, on a taped. system operated at a 
higher or lower speed than normal to produce a signal in need of 
frequency conversion. It could also occur on any type of signal 
wherein the signal is stored on a disk, record, tape, computer 
RAM memory, or otherwise in either a digital or analog form, 
which signal is capable of being operated at something other than 
real time speeds (both faster or slower) . In this modified 
device, the input signal would be reproduced at the output 
signals 1 speed in order to create a version of the second 
plurality of signals. An example of this would be to speed up an 
audio tape recorder to produce a pitch changed version of the 
audio stored thereon. At this time, the frequency converter 
system of the present application would act upon this sped up 
version of the second plurality of signals in order to modify the 
number of samples therein to meet the requirements of the new 
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clock speed and thus produce the pitch shifted output signal. No 
further memory would be needed if the sampling rate was high 
enough. In the example audio tape recorder, samples would be 
added in order to lower the pitch of the output signal to match 
the pitch of the input signal originally recorded on the tape 
recording, and thus produce a output signal having a frequency 
substantially the same frequency as the input signal. The 
frequency converter of this invention can therefore operate 
before, during or after the clock shift of the input signal. 
Similarly, the clock shift can be upwards (compression) , downward 
(expansion) , or the same as (dropout compensation, profanity 
dump) at any time during recording, storage, and/or replication 
of the input signal . 

It is envisioned that other known techniques^ such as 
homomorphic signal processing, subband coding, Fourier 
Lr n nnf n ymnH nn r r power spectrum esfeima-LiuirX, correlation, or 
measurement of relative second or higher order differences, could 
be used or adapted to improve the signal comparison. For example, 
homomorphic deconvolution, followed by linear filtering, may be 
applied to ( speech signals to separate pitch and other 
^ compoftentolj a . Separated parameters could be respectively 
compared. As another example, in subband coding a speech, image, 
or other waveform is divided into several frequency bands, where 
each band is coded separately. Individual subband components 
could be respectively compared. As a further example, spectrum 
amplitude and/or phase components as resolved by. Fourier 
transformation could be compared. In the case of correlation, 
signals could be compared over some length considerably larger 
than that of the comparison used in the present embodiment. 

It is envisioned that in an improved embodiment 
techniques such as homomorphic deconvolution, subband encoding, 
or Fourier ^ TOn^o1rmafcAon 5* could be used, with suitable 
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modification, prior to the pitch shifting operation, to provide 

signal component separation. The pitch conversion would then be 

applied to one or more of the separated signal components. As a 

particular example, homomorphic deconvolution, followed by linear 

filtering, followed by the inverse to the homomorphic 

deconvolution, may be applied to a speech signal to separate 

pitch from other components. The described invention could be 

applied to the so obtained pitch signal. Next, the frequency 

converted pitch signal would be suitably recombined with the 

other components. It is known that the representation ■ of speech 

in parametric form allows a modified pitch contour to be applied 

d isb- 
to the,da£a2\ 

It is also envisioned that data compression techniques 
would be advantageous. For example, linear predictive coding 
applied to a speech or other signals reduces the data rate 
required to represent the^wjvef ernt3 . Likewise, subband coding 
applied to speech, video, or other^ signals reduces the data rate 
required to represent t he ^ waveform?. Narrow band voice 
modulation ^iNBVWTT techniques reduce the occupied bandwidth of a 
speech signal, allowing a lower sample rate to be used. 
Operating on compressed data would reduce the described 
invention's memory and signal processing hardware requirements. 

1. Oppenheim, Alan V., and Shafer, Ronald W., Digital Signal 

Processing, Prentice-Hall, 1975. 

2. Rabiner, Lawrence R. , and Gold, Bernard, Theory and 
Application of Digital Signal Processing, Prentice-Hall, 
1975. 

3. Proakis, John G. , Rader, Charles M., Ling, Fuyun, and 
Nikias, Chrysostomos L., Advanced Digital Signal 
Processing, Macmillen, 1992. 

4. Ash, Christensen, and Frohne, "DSP Voice Frequency 
Compander for use in RF Communications," QEX, July 1994. 
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5. Benson, K. Blair, Audio Engineering Handbook, 
McGraw-Hill, Inc., 1988. 

Figure 6 show Data Manager and Program Data Processor 
implemented by a Computer Processor Unit, and Data Storage for a 
single program data stream. The data stream, which may have 
embedded information is coupled to a Embedded Information 
Recovery Circuit and to a FIFO. The FIFO is provided to 
temporarily store incoming data while the Embedded information 
(if any) is recovered and acted on by the Computer Processor 
Unit. Embedded information may contain program related 
information such as program type, air date and time, source or 
other information for the present or future pirograms which the 
user would find useful in determining the desirability of using 
or storing the program. For example, if the user desires to store 
sports highlights programs and the type of program (sports 
highlights) is carried as embedded information, the Computer 
Processor Unit will receive this information and direct the data 
storage to store the data carrying the program. When data is to 
be stored, the Computer Processor Unit selects a program address 
which the Data Storage uses to store the program. The Computer 
Processor Unit may either tell the Data Storage what address to 
use, or may receive the address used therefrom. It is preferred 
that the Computer Processor Unit keep track of all address 
information and select an appropriate address, giving that 
address to the Data Storage. The Computer Processor Unit also 
stores the information relating to the program, such as date, 
time, program name, program type, length of program, etc. along 
with the location of the program in the Data Storage, in order 
that the user may be informed of this stored information upon 
request, and in order for the Computer Processor Unit to properly 
manage storage, reading and overwriting of the data which is 
stored. The Computer Processor Unit also selects either incoming 
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data or stored data to be coupled to the MPEG decoder which 
decompresses the data stream into digital video and audio data 
streams . The video and audio data streams are then coupled to 
the User Display Interface circuit, which coupling may be direct 
or via the Time Compression and Expansion Artifact Removal 
circuit, under control of the Computer Processor Unit. The User 
Display Interface converts the digital audio and video data 
stream to proper form for use by the User Display (if necessary) 
and in addition combines user messages of audio and video type 
from the Computer Processor Unit with the data streams which are 
coupled to the User Display. The Computer Processor Unit 
controls the selection of programs to be read out of Data Storage 
by providing proper Read Program Address, and also controls the 
playback or read speed of the program in response to the users 
wishes. In the event of playback other than normal, the 
uncompressed audio and video are coupled through the Time 
Compression And Expansion Artifact Removal circuit. This circuit 
operates to remove pitch artifacts in the audio and motion 
artifacts in the video by use of the invention described in 
copending U.S. Patent Application serial number 089,904. While 
the above storage system has been described with respect to the 
preferred normal rate storage and variable rate reading, it is of 
course possible to reverse the situation with variable rate 
storage and normal rate reading in order to perform time 
compression and expansion in those systems where there is control 
of the incoming data rate. In addition, while it is preferred to 
store MPEG compressed data for reasons of memory economy, it is 
also possible to store uncompressed program data. The Computer 
Processor Unit also serves to manage the interaction with the 
User via communications with the User Remote and messages 
provided on the User Display. The Computer Processor Unit 
provides control of the Data Storage to accommodate the Users 
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wishes as well as providing and maintaining management of all of 
the data related to the stored programs and any programming guide 
information which is received via other services or the Program 
Data Stream. 

Although this invention has been described in its 
preferred form with a certain degree of particularity, it is 
understood that the present disclosure of the preferred form has 
been made only by way of example and that numerous changes in the 
details of construction and the combination and arrangement of 
parts may be resorted to to provide equivalent functions and 
elements of the invention without departing from the spirit and 
scope of the invention as hereinafter claimed. For example, the 
FIFO in figure 6 is used to allow processing of the program 
information by the CPU before accessing the data stream. This is 
necessary due to the contemporaneous transmission of the program 
information and corresponding data. With systems having advanced 
program information, this FIFO would not be necessary. Other 
changes are also possible. 
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